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ABSTRACT 



The University of Pittsburgh Program for the 
Study of Knowledge Use is now conducting 
methodological research on knovladga usa and 
school improvement undar a grant from tha , 
Nations! Institute of Education. Tha purposes 
of this working paper are (1) to identify tra- 
ditions in the study of knowledge ujie that 
have shaped the conceptual framework and 
research pletform of the Pittsburgh Program; 
(2) to outline the three mejor research prob- 
lems that the project seeks to alleviate; (3) 
to specify the research objectives and study 
design that guide project activities; and (4) 
to provide a brief overview of research pro- 
ducts and their significance. 
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INTRODUCTION 

V * v 

The University of Pittsburgh Program for the Study of Knowledge Use is 
now conducting methodological research on knowledge use and school inprove- 
ment under a grant from the Research and Educational Practices Program 
National Institute of Education. Th* Pittsburgh Prdgram, established in^b978 9 
is an interdisciplinary progrem designed to conduct research, provide instruc- 
tion, and offer technical services. Specifically, the purposes of the program 

a *? s . * 

(1) to conduct research on the production and use of research-based and 
\ «P«riential knowledge by policymakers, practitioners, and other 

\. stakeholders in issue areas ranging from education, mental health, 
v mnd criminal justice to productivity and performance improvement, 
program evaluation, and social planning and policy; 

(2) to %ffer an integrated program of instruction that promotes capa- 
cltl % t0 understand, critique, synthesize, package, and disseminate 
knowledge and skills appropriate for Improving the production and 
use of knowledge in the public and private sectors; and 

(3) to build a network of university-practitioner relationships that a 
enables a timely and appropriate response to needs for consultation, 
technical services, and training in strategies for deliberately 
enhancing the use of available knowledge to alleviate practical 
problems. 



TRADITIONS IN THE STUDY OF KNOWLEDGE USE 



\ 



The Pittsburgh Program draws from several complementary traditions in 
the study of knowledgebase. The first and most important of these is the 
tradition of applied social science established by the late Paul Lazarsfeld. 
Throughout his long and innovative career Lazarsfeld persuaded social scien- 
tists to take their cues from practice, rather than seek to build social 
science disciplines by Vjulating colleagues in the natural sciencfs (see 
Lazarsfeld, 1972; Lazarsfeld and Reitz, 1975; Holzner, Mane, and Fisher, 1977). 
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This tradition, which challenges and reverses the sequence Implied by a 
"baaic-to-yplied" research cycle, emphasizes the dependence of social 
science disciplines on the results of experience acquired in the course 
of investigating practical problems. Paraphrasing Kurt Lewin: "nothing 
is more conducive to innovation in social theory than collaboration on a 
complex practical problem" (Lazarsfeld and Reitz, 1975; 10)'. 

) A ««c<md Important tradition is the sociology of knowledge. In con- 
trast to classical sociology of knowledge, which focused on the social 
bases of intellectual productions, the Pittsburgh group is oriented towards 
a multidisciplinary social science of knowledge applications (see Holzner, 
1968; Holzner and Marx, 1979). While building upon the insights of classi- 
cal contributors (Scheler, Mannheim, Weber), a social science of knowledge 
applications is committed to studying the social bases of knowledge produc- 
tion, organization, stprage, dissemination, and use. In effect it is both 
a classical sociology of knowledge "turned upside down" "(Holzner, 1978:8) 
and a basic social science of the applied social sciences (see Weiss and 
Bucuvalas, 1980:302). 

A third important tradition is represented by the field of planned 
social change (see Zaltman, 1979). Proceeding from a recognition of 
manifest and latent consequences of change and innovation (Merton, 1949, 
1976; Rogers and Shoemaker, 1971), the Pittsburgh group has address^ 
issues surrounding the meaning and measurement of "use." Here the central 
problem is one of determining whether surface characteristics of knowledge 
use—for example, the recognition of names of authors of scientific studies 
or self-reported instances of conceptual and/or instrumental uses of know- 
ledge—actually induce intended changes in the structure and functioning of 
some social system. Thus, to take "use" at face value may blur Important 



.0 
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distinctions bstwssn knowledge inputs, outputs, end impects, risking "the 
.measurement of the number of times a bird flaps its wings without any attempt 
to determine how far it has flown" (Suchman, 1967:61). 

A final important tradition is that of methodological and empirical . 
studies of problem solving in the sciences and professions. Whereas many 
studies of scientific and professional problem solving Implicitly assume ^ 
that more knowledge will automatically alleviate problems, the Pittsburgh 
group proceeds from thev general hypothesis that most social problems are 
sufficiently complex that any one knowledge "solution" is unlikely to 
succeed unless it is produced with a view towards capturing -that complexity. 
Efforts designed to understand and Improve professional practice must there- 
fore recognize that many of the most important problems are ill-structured, 
messy, or squishy (Mitroff, 1974; Ackoff, 1974; Strauch, 1976). This means, 
among other things, that stakeholders in school improvement have conflicting 
standards for assessing knowledge-based innovations and their uses. Hany 
failures and shortcomings of the applied social sciences may be traced to a 
trained incapacity to deal with ill-structured problems (Dunn, Mitroff, and 
Deutsch, 1981). 

RESEARCH PLATFORM OP THE PITTSBURGH PROGRAM 

The Pittsburgh Program draws from these complementary traditions in a 
systematic, critical, 'and focused way. Therefore, our* aim is not to produce 
en eclectic combination of available traditions but to move towards a genuine 
synthesis. This is reflected in the interrelated methodological principles* 
which together represent our research platform: 

(1) Knowledge Use Is I nterpretive : The study of knowledge use should 
explicitly address competing interpretations of knowledge held by 
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diverse stakeholders in educational Improvement. Potsntlslly trena- 
ferrsbls knowledge products, whether research-beaed" or experiential, 
do not "speak for themselves." It is, therefore, 'essential to 
- surfece and make explicit the frames of reference which transform 
information about educational innovations into conflicting assessments 
of knowledge adequacy, relevence, end cogency (Dunn, 1981; ^eiss end 
Bucuveles, 1980; Holzner end Fishsr, 1979). The frames of reference 
of sociel scientists not only structure problems of profess ionel 
prectice (see Rsin end Schon, 1977) but distort such probllme in un- 
fortunete weys (see Gregg et el .. 1979). In effect, the fnemes of 
reference of policymakers and prectitioners contain their theor'ies- 
in-use (Zeltman, 1977), that ia, experientially grounded theories 
which provide relatively more edequete eccounts of knowledge use end 
planned change (see Dunn, 1980; Dunn end Swiercxek, 1977) than do 
formal theories developed by sociel science disciplines. 
Knowladga Usa Is Socially Conatrainad : Tha study of knowledge usa 
cannot focus solaly or axdusivaly on tha intarprativa dlntniion of 
knowladga, sinca fr etnas of rafaranca ara constrainad by social 
structuras, rolas. natworka. and othar tnatitutionalizad arrangaaants. 
Variations in knowladga usa ara • ha pad by fomal structuras. proca- 
duras, and incantiva systaoa of organisationa. "axtarnal linkagas" j 
which sat opportunitias and limits and ehas astablish a contaxt for - 
knowladga usa (Rich. 1979:333-34). Tha ralaiion batvaan intarprativa 
and social dinansions of knowladga usa is navarthalass ona of racip- 
rocity: Frantas of rafaranca affact and ara affactacf by tha social 
and political structuras in which knowladga bacomes ambaddad, insti- 
tutionallzad. /and lagitimatad. 
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(3) Knowledge Use Is Systemic : The study of knowledge use cannot confine, 

. Itself to discrete instances of knowledge use, misuse, overuse, nonuse 

and >abuse. Problems of knowledge use are rarely decomposable into 

constituent elements or parts, since knowledge use is typic&JLly a 

whole system of problems that involves interdependent. activities of 

production, organization, storage, retrieval, transfer, and applica- 

tion (Holzner and Marx, 1979). * Because knowledge systems are complex, 

and their growth cumulative, it is sometimes easier to simultaneously 

investigate a score of interdependent problems than to stiidy single 

problems sequentially. 

(*) Knowledge Use 'Is Transactive ; The study of knowledge use should not 

* > 
only build upon interpretive, social and systemic properties of know- 

ledge, but should also recognize that processes of knowledge produc- 
tion and ? use are critical social transactions. Accordingly, knowledge 
is not "exchanged," "marketed , " or "transferred," terms which suggest 
a .one-directional process, of moving discrete bits or configurations of 
information among parties who share an a priori common definition of 
"knowledge/" On the contrary, knqvledg£*is transacted among parties 
engaged in symbolic or communicative/ acts of negotiating the adequacy, 
relevance, and cogency of knowledge c^ims (Dunn, 1981b) . This 
methodological principle extends to social researchers thtmselves as 
they transact knowledge research sponsors, research subjects, and the 
wider community. Hence, the process of individual and collective 
learning is dynamic, multidirectional, and based upon self-ref lective 
or self-referential behavior imere stakeholders uncover and critically 
examine repertoires of availably standards for assessing tfre contribu- 
tion of knowledge to the alleviation ^f practical problems. 



THREE PROBLEMS OF KNOWLEDGE USE 

A continuous review of methodological needs and opportunities in the field' 
of knowledge use leads us to identify three problems which we deem yital to 
school improvement and other areas of policymaking and professional practice. 
These problems warrant special attention and urgent treatment by investigators 
in the field. ■ * 

The Criterion Problem: What is "Use?" . Traditionally, knowledge use has been 

, » 
viewed as a discrete process of appropriating new ideas or innovations for 

purposes of recommending and/or implementing specific courses of action (Na- 
tional Institute of Education, 1978) . This traditional conception of knowledge 
use, which emphasizes directly observable consequences of acting upon knowledge, 
has led many researchers to inquire whether specific knowledge is overtly used 

' and if results of implementation vary from site to site (Larsen, 1980:425).; 

With the advent of studies showing that "used" knowledge does not neces- 
sarily impact on decision making (e.g., Weiss, 1977 ;f Rich, 197?) or result in 
the solution of policy problems (fc.g., Cohen and Garet, 1975; Roessner, 1979), 
the concept of "use" has been reexamined, redefined, and differentiated. One 
broad but important distinction is between "conceptual," "instrumental," and 

. "symbolic'^ uses of knowledge (Caplan et al . , 1975; Rich, 1975; Knorr, 1977). 
More recently LAvitdn and Hughes (1981) have added the dimension of "persuasive" 
use. Similar distinctions are -contained in alternative models of knowledge use, 
for example, problem-solving, knowledge-driven, interactive, political, engi- 
neering, and enlightenment models (Weiss, 1977). Knowledge use may also be 
represented in terms of overlapping contingency models whose key element s'^are 
the content, mode of acquisition, and problem-focus of knowledge and the struc- 
tures and processes surrounding its creation, transfer, and application (Dunn, 
1980). Finally, knowledge use may be differentiated in terms of several latent 
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goals of evaluation and policy research, including legitimation, containment, 
subversion, and policing (Suchman, 1972; Rein and White, 1977). 

These distinctions accentuate the complexity of knowledge use as a concept 
and highlight the importance of attaining a better understanding of its multiple 
dimensions. At present we lack an adequate understanding of what is meant by / 
the term "use," even, though it is taken as the criterion measure in both theo- 
retical and normative models of knowledge use. Because the concept is concep- 
tualized and measured in diverse and unclearly specified ways, it 4s not now 
possible to satisfactorily explain, predict, or control "use" and its antecedent 
.^conditions and consequences. In short, little is known about the most crucial 
concept in ,th/f field. 

In the area of educational improvement this criterion problem may be 
illustrated by identifying the multiple uses of a single item of knowledge about 
teaching. A sample knowledge i£em might be that "interactive research and de- 
velopment on teaching (IR&DT)" increases in effectiveness when participating 
teachers have had similar past e xperiences with regard to innovation developments 
(TIkunoff, Ward, and Griffin, 1979:18). This knowledge may be used or not used 
in several ways: (1) educational R&D specialists may seek as participating 
teachers only those with similar past experiences; (2) school officials who are 
required' (mandated) to use this approach against their preferences may deliber- 
ately seek teachers who lack similar rias't experiences; (3) educational policy- 
n * k ?J disml8s the fl nd ln 8 because it is perceived as far tod complex to 
implement; and (4) teachers may dismiss IR&DT because the originating R&D 
teanudid not include teachers with past experiences and backgrounds, similar to 

their own. > 

* ■ . v i 

This illustration Suggests several dimensions of "use." The first case 
(educational R & D specialists) might be labelled a positive use of knowledge, 



even though the confinement of participants to those with similar experiences 
can also be considered a misuse of knowledge, for example, because those most 
in need of new experiences are excluded. The second case (school officials) 
suggests deliberate misuse of knowledge. Yet we may find that the selection 
of participants with dissimilar experiences represents a creative adaptation 
of the innovation to local circumstances. By contrast, while the third case 
(educational policymakers) suggests a simple instance of nonuse, it also 
points to jjjje dual possibility of enlightened nonuse by policymakers and 
knowledge abuse *y R & D developers who advocate innovations without satis- 
factory, knowledge of the varied conditions under which it might be implemented. 
Finally, the fourth case (teachers) involves potential adbpters who assess new 
knowledge on the basis of known uses of IR&DT by teachers like themselves. 
Teachers may have good reason to dispute the adequacy of the new knowledge 
(IR&DT), but for reasons that are experiential rather than research-based. 

These instances of use, nonuse, abuse, and misuse point to the theoretical 
and practic^importance of improving our understanding of knowledge "use": 

(1) Are there n different unidmensional concepts of use or is there a 
single construct with multiple dimensions? Or are there several 
distinct but nevertheless multidimensional constructs, for example, 
knowledge "input," knowledge "use," knowledge "output," and know- 
ledge "impact?" 

(2) How does the meaning of use vary, if at all/ when the concept ^ 
(whatever its dimensionality) is considered as an independent. 

J" 

intervening , or dependent variable? Is knowledge use properly 
conceived as an Enabling objective, a terminal objective, or both? 
Are there hierarchies of knowledge use goals and objectives? 
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(3) Do the connotations and denotations of "use" vary with the meaning 
of "knowledge" itself? Under what specific conditions are data o¥ 
evidence transformed into "knowledge?" Do different kinds of know- 
ledge— for example, experiential vs. research-b^sed and conceptual 
vs. instrumental knowledge — Imply different types and contexts of 



use? 



(4) How do the intended and actual uses of knowledge influence the * 
initiation, maintenance, and termination of knowledge transactions 
among stakeholders in educational Improvement? 
The Problem of Div ersity; Why Does Knowledge Vary in Relevance. Cogency. and * 
Adequacy? Knowledge use is fundamentally interpretive. It depends on the ' 
• diverse assumptions which different stakeholders in educational Improvement 
employ to appraise knowledge. These diverse assumptions .„ are organized 'in 
frames of reference, that is, systematically organized sets of explicit fcru^ 
implicit standards for assessing knowledge. Frames of reference involve what 
Holzner and Marx (1979:103-11) call reality or truth tests, or what Weiss and 
Bucuvalas (1980) term truth tests and utility tests. 

Truth tests are decision rules or standards for interpreting data or 
evidence (Holzner and Fisher, 1979:233-34). By contrast, utility or relevance 
tests are decision rules or standards for delineating an appropriate domain of 
inquiry or action. Finally, cogency tests are decision rules or standards for 
establishing the relative force or confidence required of knowledge in parti- 
cular circumstances (Dunn, 1981b). , 

Tests of truth, relevance, and cogency reflect diverse standards of 
assessment brought to bear in appraising educational innovations. Truth tests, 
, which are standards for assessing the adequacy of knowledge claims, are some- 
times based on scientific criteria shared by members of a research community. 
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In many other cakes truth tests are based on the ordinary knowledge of success- 
ful practitioners. Tests. of relevance, which are standard's for assessing the 
appropriateness of knowledge claims, reflect the diverse purposes attributed 
to knowledge by educational policymakers, practitioners, researchers, community 
groups, and other stakeholders. These diverse purposes underlie contrasts 
between policy research and discipline-based research (Coleman, 1972, 1980), 
"macronegative" and "micropositive" research findings (Williams, 1971), and 
the "two-communities theory" of knowledge use (Caplan, 1979). Finally, tests 
of cogency are standards for assessing the force or confidence required of 
knowledge in circumstances calling for action. For example, school officials 
may use a stringent test of . cogency in assessing results of achievement bat- 
teries which will be used to place students *±n^61a| s^a or recommend thfem for 
joljs, but apply much less stringent cogency tests when evaluating* the effects 
of established curricula on student learning. 

Concepts of truth, relevance/ and cogency raise a number of practical and 
tonceptual questions: 

(1) Do concepts of truth, relevance, and cogency exhaust the classes of 
standards used to certify and challenge knowledge claims? Can know- 
ledge adequacy, appropriateness, and force be more precisely defined 
and specified? 

(2) Can truth tests be differentiated, for example, into scientific, 
pragmatic, authoritative, and ethical rules or standards? What are 
the specific properties of these tests? Can we identify and measure 
their effects in specific contexts? 

(3) Cati relevance tests be differentiated according to specific purposes 
of stakeholders, including the description, evaluation, and recommen- 
dation of new policies and professional practices? What are the 
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specific properties of such tests of relevance and how can we identify 
and measure their influence? 

(4) Tests of cogedcy $re described in standard statistics texts in terms 
of confidence intervals and forms of statistical error (Type I and 
Type II Errors) . Are there other ways to classify and measure the 
relative force expected of knowledge in particular contexts? 

(5) . Finally, is it possible to develop a comprehensive classification 

scheme or typology which not only differentiates broad classes of 
standards *for assessing knowledge but isolates specific tests which 
are grounded in the frames of reference of different stakeholders 
in educational improvement? | 
The Multiattributef Problem; How Can We Measure the Complexity of Knowledge 
Appraisals? A typology of truth, relevance, and cogency tests assumes that 
classes of standards or rules, together with specific variants are independent. 
This assumption of independence may be tenable when the application of a rele- 
vance test does not guarantee that knowledge is sufficiently cogeAf: or forceful, 
for example, in cases where members of the legal and medical professions 
acknowledge the relevance of medical reports but apply different cogency tests, 
Th^/ legal profession typically requires a stringent cogeacy test: Defendants 
are presumed innocent until proven guilty. The medical profession, by contrast, 
typically demands a minimal test of cogency: ; treatment should err on the side 
of caution. In the first case the problem is*to avoid "false positives}" while 
in the second it is to avoid "false negatives. 11 

In many other cases the adequacy, appropriateness, and requisite force of 
knowledge interact in complex and unknown ways. Multiple and potentially con- 
flicting classes of truth, relevance, and cogency tests may be applied by the 
same person. Moreover^ the distribution and relative importance of such tests 

: '"'J 
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may change over time as a result of knowledge transactions among stakeholders. 
While Weiss and Bucuvalas (1980) have correctly observed that the complexity 
of truth and relevance tests Is analogous to problems encountered In multiat- 
trlbute utility theory (e.g., Edwards, Guttentag, and Snapper, 1975) no one has 
yet attempted to develop procedures for measuring the complexity of knowledge 
appraisals: . « 

(1) Can" the distribution of truth, relevance, and cogency tests within 

» 

; and among stakeholders be identified and measured? 

(2) Can truth tests be ranked according to their relative importance? 

If so, do they form an identifiable hierarchy? How might we measure 
the consistency of hierarchically ordered truth tests? 

(3) How does the relative importance of differerent truth tests vary as 
a consequence of employing different relevance tests? Are measures 
of consistency affected by some types of relevance tests more than 
others? 

(4^ To what extent is knowledge cogency a function of the separate, com- 
bined, and interaction effects of different truth and relevance tests? 
How does the consistency of truth and/or relevance tests affect the 
requisite force of knowledge in particular circumstances? 

(5> Finally,, can cogency tests be employed as surrogates for the value or 
utility attached to knowledge? Does the concept of knowledge cogency 
permit subjective estimates of the value of new ideas of innovations 
to educational policymakers and practitioners? Can the "shadow" value 
of an educational innovation be assessed indirectly by measuring the 
requisite force of knowledge in given circumstances? 
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RESEARCH OBJECTIVES • 

In seeking to alleviate these three problems we are guided by three 
research objectives: the development of user-responsive research instruments; 
the creation of a grounded typology of truth, relevance, and cogency tests; and 
the elaboration of procedures to conduct decision-focused utilization studies. 
User-Responsive Re search Instruments . A major objective of the project is to 
develop a set of user-responsive Research instruments to measure knowledge use. 
The development* of such research inftruments involves three Interrelated tasks: 
(D Prelimin ary Mapping of Constructs : The first task involves a prellmi- 
nary mapping of constructs that reflect multiple dimensions of know- 
ledge use. A working universe of constructs is being identified qn 
the basis of available surveys of theory and research on knowledge 
use and planned change conducted by member a of the project team. 
Conatructa are being grouped according to varlatlona in uae-related 
behavior (use, nonuse, disuse, misuse, abuse, overuse, underuse) ; . 
temporal aspects of use (preconditional use, prerequisite use, enabling 
use, terminal uae); and knowledge correlates of use (knowledge content, 
knowledge source, knowledge structure, mode of acquiring knowledge, 
adequacy of knowledge, requisite force of knowledge, relevance of know- 
ledge). 

(2) Definition and Qperationalization : This second task requires that 
conatructa in our working universe be defined conatitutively and 
operationalized in multiple formnta (self -administered questionnaires, 
open-ended and vignette-focused interview protocols, coding schemes). 
Alternative scaling and indexing procedures are now under considera- 
tion and available surveys (e.g., Weiss) and collections of instruments 
(e.g., Swierczek and Dunn, 1978) serve as potential guides. 
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(3) PUot-taatlng and Provision*! Validation ; Instrument* will be pilot- 
tested among selected educational policymakers and practitioners in 
Pittsburgh. Respondents will be chosen by attempting to maximi ze 
differences between formal positions, roles, functions, and involve* 
ment in various phases of the process of adopting educational innova- 
tions. Instruments will be pilot-tested in several ways, for example, 
by selecting stakeholders in the same events (events sampling) , 
similar social networks (sociometric sampling), and connon phases of 
the adoption process (phase sampling). Pilot-test results will be 
provisionally validated with appropriate procedures (e.g., face 
validation, factor validation, contWt validation, concurrent vali- 
/ dation, construct validation). Since our objective is confined to 

instrument development issues of external validity are not directly 
•relevant. 

A (Grounded Typolofty. The second main objective is the development of a grounded 
typology of standards for appraising knowledge. These standards- in-use are 
based on a preliminary classification of truth, relevance, and cogency tests 

developed by members of the research team. The typology is being developed at * 

* 

the same time as we complete the first task , of instrument development (Prelimi- 
nary Mapping of Constructs), since constructs, of knowledge adequacy, relevance, 

and cogency are central to the .conceptualization and measurement of "use." 

f 

Elements of the typology will be defined, operationalized, pilot-tested, and 
validated as part of the latter two tasks of instrument development described 
above. 

Peels ion-Focused Analytic Procedures . The third main objective is to develop 
procedures appropriate for analyzing the complexity of diverse standards for 
appraising knowledge. The focus of these analytic procedures is the use of 
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knowledge to make various classes of decisions. In the tradition of Dewey and 
others (see O'Shaughnessy, 1973; Dunn, 1981a) we do not limit decisions to those 
which Involve action but Include: (1) decisions to denote, connote, and clas- 
sify (definitive decision); (2) decisions to describe, explain, and, predict 
(designative decision); (3), decisions to evaluate (evaluative decision); and 
(4) decisions to recommend a course of action «r_gplicy (advocative decision) . 

In analyzing such decisions we use the idea of frsmes of reference as a 
unifying construct. Truth, relevance, and cogency tests— three classes of 
components of frames of reference—are treated as multiple attributes of know- 
ledge affecting decisions. The main task is to identify the presence of these 
attributes,, rank them in order of Importance, and measure their weighted and 
unweighted aggregate "decision" value to different stakeholders. For these 
purposes we are considering appropriate ways to adapt and apply multiattribute 
procedures used in marketing (e.g., Green and Srinivasan, 1978), program evalu- 
ation (e.g., Edwards, Guttentag, and Snapper, * 1975) , and other areas involving 
judgments about the utility of information (see Slovic and Lichten-Stein, 1971). 
In addition we are. assessing the appropriateness of analytic hierarchy analysis, 
a unqiue set of procedures used to scale educational planhing priorities and 
*Mly«e^the complexity of various hierarchies of attributes (Satty and Rogers, 
\WS\ Saaty, 1977). Finally, we have begun to explore the extent to which 
certain concepts and procedures borrowed from applied economics and statistical 
decision theory may be applied to problems of estimating the use value of know- 
ledge. These concepts and procedures include shadow pricing and minimax regret 
(see Savage, 1951). 

* STUDY DESIGN 

To achieve these research objectives we have developed a study design with 
three distinguishable but interdependent components^ Since the central thrust 
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of the pro J tct la methodological, thaaa daaign componente ara aim ad at the gener- 
ation of concapta and meaaurae of knowladga. Only eecktf&arily do va aaak to 
validate raaulta through fiald rejkarch. \ 
Multiple Triangula t Ion . Tha f Irat daaign compenent la intended to provlda an a vera 
to tha quae t ion: What la knowladga M uaa? M Anawara to thla queatkon ara vital to J 
tha raaolutlon of policy laauaa eurrounding knowladga uaa and ecKfel improvement, 
alnca tha way wa daflna "uie" datarmlnaa conclualona about tha effectiveneae, 

A 

efficiency, adequacy, and equity of fadaral invettmente In aducational R&D ef forte. 
If "uia M \a daflnad aolaly In tarma of tha ovart and actlva appropriation of 
knowladga to allavlata aoma aducational need, than many aducational R&D ef forte 
ara llkaly to ba aaaaaaad aa marginally worthwhile or even aa failuree. Tat 
auch negative conclualona may be challenged on grounda that there are other 
equally Important deflnltlona of uaa (Zaltman, 1980). Fot example. M uae M may be 
defined aa: 

(1) Adaptation : New knowledge la modified In reaponaa to apeclal problane 
or circumetancea preaant In tha environment of the uear. For example, 
achool diatricta may adapt tea ting procedurea to local condltlone. 

(2) Segmentation ; New knowledge may be aagmented Into parte, aoma of 
which may ba uaed and o there rejected, aa la tha caae with many manage- 
ment innovationa. 

(3) Sedimentation* New knowladga may appear to be wholly Ineffectual, 
aince potential uaara can neither cite nor deacrlbe it In ita original 
form (for example, a key aclantlflc report or atandardlsed ay a tern of 
ine true t ion) . In fact, parte of auch knowledge have paaaed through 
auccaaaiva phaaea of tranalatlon in the field of practice and eedimen^' 
tal uaara ara unaware of the original aource of their knowledge. 
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(4) Rejection; New knowledge may be rejected, but nevertheless used in a 
-Productive W to critically test jt,nd improve existing practices. Re- 
jection may involve a kind of a fortiori ("with all the greater 
strength'^) logic where' a nav but intuitively disfavored knowledge source 
if systematically compared with an existing source which eventually 
withstands critical comparison and is also improved in the process. 

(5) Displacement: New knowledge in the form of negative research findings 
("X does not work") may displace current practices and promote the -, 
development of revised or Improved local solutions, even though nega- 
tive findings do not themselves contain knowledge which may be "used" 
td^ take direct action to resolve a problem. 

(6) Synthesis; New knowledge Jaay be juxtaposed to current practices, \ 
creating additional new knowledge that cannot be described in terms of 

' t *»« N priginal juxtaposed components. The resultant synthesis goes 
beyond dlchotomous choices between the adoption of new knowledge (K) 
i •» and the maintenance of current practices (not-K) and represents a new 
form of use (notrK and not-not-K) . 
These alternative meanings of "use" pose a number of conceptual and instru- 
mental problems whose resolution is critical for knowledge use and school Improve- 
ment. If "use" Is measured by the frequency of knowledge products cited by 
educators as important to decisions made within their respective domains of 
responsibility, then we will" no doubt find that the relative frequency of "use" 
and "nonuse" Is heavily skewed In "the direction of the latter category. Even 
when continuous scales are employed to measure multiple dimensions of use, these 
scales are not likely to provide an accurate representation of adaptation, seg- 
mentation, sedimentation, rejection/displacement, and synthesis. For example, 
'Larsen (1979) has developed a Utilisation Quotient (UQ) of the form 



18 



OQ - S(St x I 



N 



x 3c) ; where 




St - the lmplementational status of information utilisation as rsprsssntsd 

by its rslativs incorporatidn into dscisions; 
I ■ ths influsncs of ths knowledge providsd in affecting utilisation;, and 
Sc ■ tha scops of rssourcss required to uss information 

Tha valuas of variables in this 1 simple (unweighted*) iggragativa index ara 
obtainad by using tsn-point respondent raOngJteaffss. Ths problem Is that raspon- 
dants with identical or highly similar UQ scorss nay have "usad" knowledge in 
markedly different ways. A practitioner with a UQ scors of>ss ay, 240 may hav e 
successfully adapted knowledge to the local context, while another practitioner^ 
with a UQ score of 245 may liava segmented or synthssissd knowledge. The Utili- 
zeeton Quotient (UQ) and similar measures, even where they ara subj acted to 
rigorous -internal consistency reliability routines and procedures for construct 
validation (Nunnally, 1967) will not necessarily reflect the real but unknown 
meaning of "uss." Indeed, this particular problem is characteristic of every 
pubUehed- empirical study of knowledge uss of which ws ara aware (e.g., Caplan 
et al. , 1975; Rich, 1979; Weiss and Bucuvalas, 1980). V 

A sscond major methodological problem is connected with the time at which 
"uee" is measured. U we conceive of knowledge uss as a process of planned 
social change (ZaltaanV 1979) wa can devise various seneitising concepts which 
togsthsr point towards different "phases" of knowledge uss (sss Zaltman and 
Duncan, 1977; Rogers end Shoemaker, 1971). A framework outlining potential 
ralationahips among these sensitising concepts is presented in Figure 1, which 
shows the possible routes that knowledge may travel in reaching its destination 



es "uss." 



(Place Figure 1 about here) 
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Figure 1 - 
KNOWLEDGE USE AS PLANNED CHANGE 
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The sensitizing framework presented in Figure 1 depicts a* linear process 

r 

which, in fact, may be "short-circuited" at any point. Moreover, the validity 
of constructs designating component "phases" is heavily dependent of temporal 
properties of knowledge use. Thus, for example, if "use" is prematurely mea- 
sured at the legitimation phase, then researchers- will underestimate the scope 
and level of use. If the appropriateness of sampling strategies is determined 
by characteristics of the phase at which use is measured, sampling procedures 
should so reflect the complexity of this phase of change. For example, the 
problem percepfiion phase often involves diffuse networks of providers, trans- 
lators, linkers, and potential users. These diffuse networks cannot be 
satisfactorily represented by probability samples that take individuals as 
units of analysis; sociometric or snowball sampling is more appropriate. Prob- 
ability samples taking individuals as units of analysis nevertheless may be 
wholly appropriate at the motivation pf»ase, where networks of stakeholders who 

affect and are affected by the problem have already been identified. Finally. / 

> * / 
it is essential to recognize that knowledge is "used" not only at the "terminal" 

phase— for example, to adopt, adapt, segment, or synthesize knowledge—but also 
at "enabling" phases. Thus, knowledge is often used to structure or restructure 
a problem, processes that may eventually result in knowledge displacement or 
knowledge synthesis: 

These and related questions about the meaning and temporal properties of 
"use" are closely tied to different approaches to the study of knowledge use. 
Contr as ts : ,among"appr baches involve potential choices among several different 
classes of units of analysis: products, persons, issues, organizational con- 
texts, ana" phases (compare Weiss, 1979). Each of these classes and their 
subdivisions (Figure 2) imply different methodological .strategies for studying 
knowledge use. 
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* Figure 2 i * 

UNITS OF ANALYSIS AND IMPLIED METHODOLOGICAL 
STRATEGIES FOR STUDYING KNOWLEDGE USE 
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ties and the multiple meanings attached to "use" by stakeholders in educational 



improvement, tor this reason we believe that the most appropriate strategy for 
research is one that combines a concern with multiple operationism (Webb et al. , 
1967) with that of generating constructs which are grounded in theories-in-use 
about knowledge held by different stakeholders (see Glaser and Strauss, 1967; 
Dunn and Swierczek, 1977; Zaltman, 1977). 
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In following a strategy of multiple trlangulatlon (Denzin, 1970; Si|lth, 1977) 
we will attempt to identify, through available studies, reports, records of exper- 
ience, consultants, and other key informants the .multiple theories-in-use of 
educational policymakers and practitioners. This will not only permit the devel- 
opment of multiple constructs of "use" and their underlying dimensions, but will 
suggest appropriate multiple methods of acquiring data, multiple temporal frames, 
and multiple contexts. Specifically, this design component involves the mapping 
of multiple constructs of "use;" the translation of these constructs into consti- 
tutive and operational definitions of "use;" and the development, pilot-testing, 
and provisional validation of instruments for measuring knowledge use and school 
improvement . 

Conceptual Elaboration . Answers to questions about the meaning of knowledge use 
cannot be obtained without also providing answers to a second question: Why does 
knowledge vary in relevance, cogency, and adequacy? This question is central to 
all questions of knowledge use, since it points towards the variability of stan- 
dards for appraising knowledge itself. While there are many ways to explain such 
standards, we believe that effort? to specify social bases and structural corre- ' 
lates of knowledge and its use are premature. Indeed, it is a reasonably simple 
matter to develop hypotheses 'about the effects of variations in social organiza- 
tion on knowledge use, for example, by turning to theories of complex organizations, 
bureaucratic politics, social networks, and so on (see Weiss and Bucuvalas, 1980). 
The challenge lies, rather, in obtaining a better understanding of the "knowledge" 
in knowledge use. 

Parallel to the development of research instruments .are elaborating concepts 
bearing on diverse standards of knowledge adequacy, relevance, and cogency. In 
elaborating concepts we proceed from the position that educational Improvement 
and other types of social reform are best viewed as arguments or debates which 
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Involve critical social transactions among different stakeholders (Dunn, 1981b). 

The use of argument as a unifying construct is based on prior theory and research 

on the logic of practical discourse (TouJmin, 1958; Toulmin. Rieke, and Janik, 

1979), the functions of forensic social science (Rivlin, 1973; Brown, 1976) , and 

the uses of argument in educational evaluations (House, 1980). * 

One of the strenghts of argument as a unifying construct is that it moves 

us away from assumptions that knowledge use is solely or primarily governed by 

standards of scientific adequacy and relevance. Instead, the model of argument 

points to the Importance of studying multiple standards of appraisal in contexts 

of professional practice. In school settings there ^are multiple frames of ref- 

erence-what Tyler (1978) prefers to call "cognitive maps»~which contain diverse 

standees for appraising outcomes of educational research and development: 

New research is likely to produce new concepts and generalisations 1 
or give different emphasis to old ones. Practitioners may hear 
about them or read about them, but they are not* likely to incorpor- 
ate them into their maps unless they meet one or both criteria- 
consistency with the reality they perceive, and/or providing gui- 
dance for more effective performance of their tasks. The Criteria 
of Internal consistency and parsimonious explanation, which are 
Important to researchers, appear- to have little significance to 

r^ a «I C !ii "T r8 \f ld , thl * U ° ftMl the re " on "—"h 
reports are given little attention by practitioners and evoke no 
changes in their behavior (Tyler, 1978:97). 

In elaborating tests of knowledge adequacy, relevance, and cogency we pro- 
ceed from a provisional framework already developed by members of the research 
team (Dunn, 1981b). This framework classifies disputed knowledge claim* jLnto 
four types of potentially relevant uses: definition, designation, evaluation, 
and advocacy. These four type, of potentially relevant uses, based on work in 
forensic, and .eoiotic («ee Brockriede and Ehringer, 1960), may be cross-classi- 
fied with types of standards for evaluating the adequacy of knowledge. Major 
types of truth tests are empirico-analytic, interpretive, pragmatic, and authen- 
ticate in character. Finally, we hypothesize that the adequacy of knowledge 
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does not ensure its relevance; nor is the reverse true. Even where knowledge is 
both adequate and relevant it may lack the requisite force expected in given 
practical circumstances. Relations among tests of adequacy, relevance, and cogency 
may be visualized in terms of a factorial design, where given levels or threshholds 
of cogency are a function of the main ^nd interaction effects of knowledge adequacy 
and relevance (Figure 3). » 



Figure 3 
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SOURCE; Dunn (1981b); Holzner and Fisher (1979). 

In elaborating a typology of truth, Relevance, and cogency teats we are 

i 

making use of available theory and research on knowledge use and planned chatage 
in education and other areas. In addition we are employing consultants and other 
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key informants, both in personal discussions, interview setting/, and in workshops. 
This provides a vehicle for grounding the typology in the theories-in-use of edu- 
cational policymakers and practitioners. Finally, the typology is an integral 
part of the development of instilments to measure knowledge use. As such, ele- 
ments of the typology are included in research instruments. 

Vignette-Focused Simulations . This third design component provides answers to 
the question: How can we measure the complexity of knowledge appraisals? Answers 
to-questions about the meaning of knowledge and its uses cannot be satisfactorily 
provided without squarely addressing the coexistence of multiple interdependent 
standards of knowledge appraisal. Hence, even if we know how to conceptualize 
and measure knowledge and its varied uses we still require analytic procedures 
which bring out the underlying or latent structure of frames of reference and 
their constituent elements. 

This design component focuses on the simulation of decisions to use know- 
ledge. The problem here is analogous to that of decision- theoretic evaluation 
research, insofar as decisions to use knowledge are based on multiple attributes 
(Edwards, Guttentag, and Snapper, 1975). For this reason it is appropriate to 
construct vignettes which describe different combinations of knowledge contexts, 
contents, and alms. Scales and indexes' can then be devised to measure variations 
in adequacy, relevance, and cogency attributed by stakeholders to knowledge 
described in vignettes. Vignette-focused simulations will be carried out in the 
form of personal interviews with consultants. Results of pilot- tests may be 
analyzed with several alma in mind: 

(1) the assessed cogency of two or more knowledge claims in resolving a 
problem may be used as a basis for estimating the "opportunity loss" 
of knowledge {Savage, 1951); 

(2) the cogency of a single knowledge claim may be used as a surrogate or 
"shadow" value of knowledge in particular contexts; 

•ERIC , 33 . 
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(3) the relative Importance of different standards of adequacy and/or 
relevance may be uncovered through binary choice procedures; and 

(4) weighted and unweighted aggregative "usability indexes" may be con- 
structed on the basis of individual and group responses. 

This third design component, while it poses special problems and opportuni- 
ties, is being closely coordinated with the development of user-responsive in- 
struments and the creation of a grounded typology of truth, relevance, and 
cogency tests. \* 

RESEARCH PRODUCTS AND THEIR SIGNIFICANCE 

The proposed study design wili culminate in three major products, each of 
which is closely attuned to unresolved problems of knowledge use and school 
Improvement. We hope that these products' will make a substantial contribution 

to basic knowledge about school improvement efforts and to foundational studies 

j 

of knowledge use generally. 

Measuring Knowledge Use: A Handbook ot Research Instruments . One of the main 
products of the project is a handbook Of research instruments for measuring know- 
ledge use. The first part of the handbook will provide a systematic review of 
conceptual issues surrounding the measurement of knowledge use. The second part 
Will review available research instruments and those developed in the course of 
the project. The format of this second part will contain elements found in other 
handbooks of measurement, for example, Lake ahd Miles' Measuring Human Behavior 
(1974). The handbook will provide Improved concepts and measures of how teachers, 
principals, and other educators use knowledge in their daily work; facilitate an 
improved understanding of the motivations of educational policymakers and practi- 
tioners as they use knowledge for school improvement; and outline new ways of 
identifying and measuring the many ways that knowledge resources affect and are 
affected by different phases of the clpange process- 
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1 Understanding Knowledge-In-Use: A Concept Paper . The second main research pro- 
duct is a concept paper, provisionally titled "Understanding Knowledge-In-Use," 
which outlines the typology of standards for appraising knowledge. The concept 
paper is likely to be divided into * short introduction, an overview of the 
typology, and additional sections dealing wfth tests of adequacy, relevance and 
cogency. This concept paper will provide a conceptual foundation for major 
portions of the handbook and identify the range of criteria employed by educa- 
tional policymakers and practitioners to assess the truth, relevance, and cogency 
of new knowledge. 

A Decision-Focused .Approach to Kn owledge Use: A Technical Paper . TJp third main 
product is a technical paper which elaborates a decision-focused approach to the 
study of knowledge use. The technical paper will systematically present steps 
required to analyze information and obtain summary measures and indices of know- 
ledge use. The technical paper will provide explicit and replicable procedures 
for uncovering and ranking multiple criteria employed by educators to evaluate 
knowledge;- and facilitate the measurement of potential changes in these multiple 
criteria. Investigators have recently outlined the contours of a similar approach 
in"^he area of performance measurement (Dunn and Mitroff, 1980). 
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ABSTRACT , L . 

This bibliography is based 4 on a systematic 
search of the literature and contains 112 
annotated citations of recent work in the 
4rea of knowledge utilization within the * 
field of education. * Each item is annotated 
according to one <of JEiv* categories: (1) 
Provides insight into the KP&U process in 
education; (2) Identifies factors which 
impede KO iq, education; (3) Suggests strat- 
egies for Improving KD in education; (4) 
Concern with theories of KD in education; 
and (5) Attempts to define the dependent 
variable M use. M 
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C^S INTRODUCTION 

This annotated bibliography*attempts to provide a collection and summary 
of recent ^ork that has been done in the area of knowledge utilization within 
the field of education. An attempt was made to include the major contributions 
to theory and research within these fields. Included among the annotations are: 
theoretical analyses challenging the underlying assumptions regarding the rela- 
tionship between educational research and policy, empirical studies identifying 
.factors that affect knowledge use in educational contexts; strategies to improve 
the usefulness of research for educational policy; and case studies which ana- 
lyze the ways in which research findings were used in specific educational 
settings* 

Ther6 were several purposes for compiling this annotated bibliography. 
First to identify the relevant literature and Important issues in the area oi 
knowledge use in education. Second, to determine the problems of knowledge use 
that are specific to the field of educationT. Third, to determine the nature of 
the relationship between educational research and educational policy making and 
school improvement. Fourth, to identify gaps in the knowledge use/school lm- 
provement-J^terature. 

SELECTION OF ITEMS 

The items selected foiv inclusion in the bibliography all deal substantively 
with the field of education. In addition, the items generally can be grouped 
into one of five categories: 



* The author gratefully acknowledges the assistance of Ralph Bangs, Jan Benda, 
Tom Hclntyre and Hassan Rahmanian in assembling this document. 
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1. Provides insights into the underlying process of knowledge production 
and utilization in education. Knowledge utilization in education 
is a complex process which is affect&i by many factors which are 
peculiar to education. Understanding knowledge use in education 
depends upon understanding the educational system and the users 
(teachers) within the system. Lortie (1975) provides an excellent 
characterization of the pattern of orientation and sentiments which is 
unique to teacher* and distinguishes them £rom members of other occupa- 
tions. In examining the educational system in our society,, Boyd (1978)' 
outlines the nature of the political problems which are created by the 
need for- the institution of education to simultaneously maintain society 
while responding to pressures fd£ societal change. 

Other authors examine the nature of the relationship between educa- 
tional research and school improvement. Some authors contend that 
usefulness will always be a prqblem with educational research because 
it is a function of the different contexts and methods that character- 
ize the research and policy functions (Levin, 1978; Pellagrin, 1965) . 
Other authors examine and challenge the aissumptions currently under- 
lying the relationship between research and policy in education (Cohen 

V 

and Garet, 1975; Coleman, 1978). Some of these authors suggest new 
ways of understanding, thinking about aad doing research on knowledge 
use and educational change (Berman, 1981). 
2. Factors which impede knowledge use in education. Throughout the know- 
ledge use/school improvement literature, a diverse set of variables have 
been identified as potential facilitators or impediments to the use of 
social science research for the purposes of improving educational prac- 
tice. These factors' can be grouped into three broad categories: 
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a) Individual/Personal. This category attributea innovation adoption 
to variables which deal with personal factors with the Individual 
as the unit of analysis. These variables Include social psycholo- 
gical factors (Schmuck, 1968) , the importance of f ace-tb-f ace 
contacts and social relations (House, 1974; House, et al. , 1972), 
and the role of the principal In facilitating Innovation In 
schools (Chesler, Schmuck an, Lippitt, 1963) . One author emphasizes 
the- Importance df subjective 'criteria "that Individuals use to 
evaluate the usefulness of an Innovation. Doyle and. Ponder (1978) 
refer to these criteria collectively as the practicality ethic . 

b) Product.* This category attributes innovation adoption to variables 
related to the research product (Bissell, 1979). More specifically, 
some authors link Innovation adoption to the quality of research 
(Levin, 1978; Pellegrin, 1965; Dickey, €980), or the way the pro- 
duct is written (Levin, 1978).- 

c) Structural. This category attributes innovation adoption to vari- 
ables uniquely related to educational organizations. Features of 
the public education system which are unique Include goal ambiguity, 
weak connections between activities and outcomes, political vulnera- 
bility (Sieber, 1968? Deal, 1979; Miles, 1965; Zaltman, et al., 
1977) and its incentive system-j(Pincus, 1974; Sieber, 1981). 

3. Strategies for Improving knowledge use in education. The literature 
reviewed for this bibliography suggest three major strategies for im- 
proving the usefulness of social sciejice research to school Improvement 
efforts. The first strategy Involves the integration of "qualitative 
and quantitative methods (Madey, 1981; Corbett and Firestone, 1980). 
It is believed that . such an integration would improve evaluation de- 
sign, data collection and analysis (Madey, 1981). Another strategy 
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prevalent in this literature Is organizational development (Fullan, 
Miles and Taylor* 1980). Much of the literature Suggests, however, 
that OD (as it currently -exists) has failed to have a far reaching 
impact in educational change (Blumberg, 1976; Derr, 1976). A third 
strategy for facilitating educational change in schools is the use of 
linking agents. A general review of this literature can be found In- 
Hood and Cates (1978) . A review which includes an analysis of gaps 
and deficiencies in research regarding the roles and functions of 
external agents can be found in Louis (1981). Other authors discuss 
the types of linker roles (Havelock, 1968), the problems facing link- 
ing agents (Crandall, 1977) and whether linkage agents should be 
internal or external to the drganization (Hayman, 1979). 
Theories of knowledge utilization in education. In the field of 
educational change and knowledge utilization there are a number of 
different approaches and conceptual schema for achieving knowledge 
utilization in education. Several authors have reviewed and critiqued 
these existing models and have suggested alternative models (Fullan, 
1972; Zaltman, et al., 1977; Sleber, 1968, 1974; Paul, 1977). Several 
authors have attempted to develop theories of knowledge use in the 
educational context. Dickey (1980) and Alkin and Daillak (1979) both 
have examined evaluation utilization and have suggested "theories" of 
use. In a different context, legislative decision making, Mitchell 
(1978; 1980) and Florio, et al. (1979) have attemtped to develop 
.theories of knowledge use, taking into consideration the different 
stages involved in legislative decision making and the different inforr 
mation needs of the different stages. 

Defining the dependent variable "use." Defining the dependent variable 
"use" is one of the most difficult yet challenging tasks currently 
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facing researchers in the area of knowledge use. Several authors have 
attempted to identify the types of uses and subuses of knowledge in 
educational, administration (Culbertson, 1977) whil/some have equated 
use with implementation or actual use of an innovation (Fullan and 
Pomfret). In addition, some authbrs have examined the effect of J 
research on the development of public policy. . Florio (1978) discusses 
how research information is mixed with value choices in public policy 
decisions. In a slightly different context, Berne and Stiefel (1979) 
have identif led and illustrated five perspectives for examining the 
impact of research on school finance policy. However, the majority of 
' .authors concerned w4|h defining knowledge utilization in the field of 
education have focused on describing ways practitioners use research 
• (White, 1979). Tyler (1978) has suggested that practitioners commonly 
use research to help form the conceptions that make up their cognitive 
maps. The most sophisticated work in this„area has been conducted by 
Hall, et a)., at the R&D Center for Teacher Education, The University 
of Texas at Austin. Hall et al. (1975) have identified and described 
eight discrete levels of use of an innovation (LOU) . 

SEARCH PROCEDURE 

There are any dumber of ways a literature search similar to this could be 
compiled. This literature search was conducted at the University of Pittsburgh 
in several stages: 

Stage One. Consultation with content Experts in Education and knowledge 
use. This was helpful in becoming familiar with key authors, 
books, and journals in the field. 
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Stage Pour. 



Stage Two. Use of the Social Science Periodical Index and the Social 

Science Citation Index. Both of these Indices allow the user 
to search for citations by key words and authors. 
Stage Three. Use of the card catalog In order to determine what types 'of 
books (particularly books of readings) existed on this topic. 
Current and Back Issues of Education and Education related 
Journals. A list was compiled of these journals, and the 
table of context was skimmed for all current and back issues 
of these journals. Relevant titles as. well as authors were of 
Interest. 

Existing Bibliographies . NIMH in conjunction with the Hunan 
Interaction Research Institute (1971, 1976) has compiled at 
least two annotated bibliographies of the literature regarding 
Knowledge Transfer and change. 

An automated bibliographic search was conducted through the 
Social Sciences Information Utilization Laboratory (SSIUL) at 
the University of Pittsburgh. This search includes items . 
which have appeared in the Current Index to Journals in Educa- 
tion (CUE) between 1969 and present. CIJE is produced by the 
Education Resources Information Center (ERIC), an agency of 
the National Institute of Education. 



Stage Five. 



Stage Six. 



UPDATING PROCEDURE 

This bibliography Is an ongoing project of the NIE Project Team. The reader 
Is- asked to participate In enhancing the value of the bibliography to users by 
making suggestions for Its Improvement and/or suggestions regarding additional 
citations to be Included as the bibliography Is updated. 
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Abbott, Max G. (1964) "The Schools Social System: Indicators for Change," Socio- . 
economic Planning Services . 2 (1969) : 167-174. 

This paper outlines those properties or attributes of the school organization 
that will increase the probabilities of bringing about deliberate and planned 
change. As a subsystem, school organization maintains a viable relationship with the 
broader social system through adaptation process. In order to analyze the process 
by which innovations are introduced into a school and, to relate that process to 
organizational attributes and properties; three conceptual phases, are identified: 
(1) awareness phase; (2) search phase; and (3) Implementation phase. For each 
phase, the paper derives some suggestive propositions and hypotheses from the 
generalizations about organizational change based on current organization theories. 



Alkin, Harvin C. and Richard H. Daillak (1979) "A Study of Evaluation Utilization," 
Educational Evaluat ion and Policy Analysis . Vol. 1, No. 4, 41-49. 

This article is an attempt to formulate a theory of evaluation utilization. 
Case studies were used to see what uses were made of the evaluation; to document 
the extent of the evaluation and to trace the evaluation process -in detail; to 
identify any strategies employed by the evaluator; and to examine the forces 
which Shaped the eventual outcome of the situation. The findings of the study 
indicate that modifications in the programs , as a direct result of the evaluati 
were rare; what was more common was gradual, incremental impact. The case 
studies show that evaluation utilization does often occur, though seldom in 
major ways. The authors suggest a framework which focuses on those facets n 
critical to utilization. The framework consists of eiglit^categories which 
affect utilization: 1) Preexisting evaluation bonds; 2) Orientation^oflth* 
Users; 3) Evaluator' s approach; 4) Evaluator credibility; 5) Organizational 
Factors; 6) Extraorganizational Factors; 7) Information Content and Reporting; 
, 8) Administrator Style. The authors consider this to be an important first 
step in understanding evaluation utilization. 



Allan, G.S. and W.C. Wolf (1978) "Relationships Between Perceived Attributes of Inno- 
vations and Their Subsequent Adaption," Peabody Journal of Education . 55, 4:332-336. 

This study examines relationships between educators' innovation adaption beha- 
vior and five attributes of innovation described by Rogers and Shoemaker, i.e., 
relative advantage, complexity, comparability, trialability, and observability. 
These attributes are treated as independent variables and* adoption of innovations 
is considered as dependent variable. The results of these an%yses only support 
the hypothesized relationship concerning complexity of innovation and its adoption. 
The other. four hypotheses are rejected in the context of this_study. 
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Armstrong, D.G. (1980) "Researchers ■ Incentives end the Dearth of Practical Research 
Results," Peabody Journal of Education , October ; 55-60. 

This article advocates the challenge of providing researchers with real incen- 
tives to produce quality research with direct applications to the classroom. 
Unless teachers generate a real demand for products of research as opposed to a 
cosmetic danand, researchers will continue to respond to incentives coming from 
> administrators, from a process-oriftated national research community, and from 
tenure and promotion panels. These existing incentives stimulate the production 
of studies large in number, limited in scope, and deficient inequality. Educa- 
tional researchers interest^ in providing research issues of genuine import to 
classroom teachers face an imputative of directly acting to modify the system of 
incentives undergirding present research practices, 
(author summary, modified) 

# 

Baldridge, J.V. (1975) "Organizational Innovation; Individual, Structural and 
Environmental Impacts," in Managing Cha nge in Educational Organization. . (eds.) 
J.V. Baldridge and T.E. Deal. Berkeley: McCutchan Publishing Corporation. 

This article argues that traditional research on innovation and organizational 
change has focused on the .wrong sets of variables. More specifically, an orien- 
tation toward the early phases of the innovation cycle, preoccupation with small- 
scale technical innovations, and individualistic biases have hindered our under- 
standing of major organizational innovation. The author suggests that a more 
productive analysis of the change process should focus on complex technologies 
with ambiguous evaluations, shift emphasis from individualistic variables to 
roles and organizational structure and investigate environmental factors more 
closely. The second half of the article presents an overview of results from two 
research projects on organizational change. The results in both studies support 
the hypothesis that a large, complex organization with a changing heterogeneous 
environment is likely to be more innovative than a small simple organization with 
a relatively stable, homogeneous environment primarily due to the "demand struc- 
5" 18 t#1 * ted t0 • lze of th « organization. The findings also indicate 

that differentiation and structural complexity are critical for innovation. 



Ben-Peretz, M. and Pinches Tamir (1981) "What Teachers Want to Know About Curriculum 
Materials," Journal of Curriculum Studies , 13, 1:45-54. 

The purpose of the research reported in this article was 'to investigate the 
following questions about the views and concerns of Israeli secondary teachers 
in relation to centrally produced curriculum materials: 

1) What questions about curriculum materials concern teachers most? 

2) What is the relationship between the importance teachers attach to these 
questions and their preferences regarding actual curriculum characteristics? 

3) What Image do teachers have of their own role in curriculum Implementation? 
The author's findings are discussed in terms related to teacher attitudes regard- 
ing their professional roles and competencies. 



ERIC 



i 



Beroan, P. (1981) "Educational Change: An Implementation Paradigm," in Improving 
Schools: Using What We Know , (ed.) R. Lehming and M. Kane. Beverly Hills: Sage 
Publications. 

This paper contends that the accumulation of empirical studies point toward 
new ways of understanding, thinking about, and doing research on knowledge utili- 
zation and educational change. The focus of the paper is on the idea that the 
very inconsistency of research findings over the last decade may reflect educa- 
tional reality, not ^gmply inadequate methodology. This paper reviews research 
on educational change in the 1950* s and 60* s in order to identify the predominant 
research paradigm of the period. This •paradigm is called the Technological- 
Experimental paradigm. Furthermore, the paper describes the nature of the para- 
digm shift by proposing three meta-propositlons that encapsulate a new emergent 
paradigm. The three propositions in new paradigm are: (1) implement at ion-domi- 
nant proposition; (2) loosely coupled process; and (3) the conditional nature of 
explanation and prediction. This new paradigm offers no guarantee of the eventual 
emergence of a unified theory of educational change. Instead it accepts lowered 
expectations fofc research findings. This paper concludes with some reflections 
on future research. $ 

Berman, P . , * McLaughlin* M.tt., et.al . (1975 and 1977) Federal Program* Supporting 
Educational Change (8 volumes). Santa Monica, CA: Rand Corpotation, 1975 
(volumes 1-5);, 1977 (volumes 6-8). 

This series of five reports describes the results of the first year of the 
Rand Change Agent Study (July 1973-July 1974). Volume I (A Model of Educational 
Change) provides a theoretical perspective for the Rand study by analyzing the 
current state of knowledge of planned change in education and by proposing a 
conceptual model of factors affecting change processes within school districts. 
Volume II (Factors Affecting Change Agent Projects) contains the analysis/of 
survey data collected from a national sample of 293 projects iij 18 states. 
Volume III (The Process of Change) summarizes the results pf 29 case studies of 
change agent projects conducted in 25 school districts. Volume IV (The Findings 
in Review) summarizes the findings of the previous volumes land also synthesizes 
extensive data collected on federal level program strategy and management for 
each of the change agent programs. This volume also discusses alternative 
strategies for promoting innovation. Volume V (Executive Summary) presents the 
study's method* and results. Volume VI (Implementing and Sustaining Title VII 
Bilingual Projects) discusses the complex process of establishing bilingual 
programs in local school districts, with particular attention given to those 
aspects of the Title VII program and to those political influences that affect 
local implementation. Volume VII (Factors Affecting Implementation and Continu- 
ation) presents an analysis of the survey data collected in! 100 Title III pro- 
jects in 20 states. This volume deals specifically with thk questions of 
implementing and sustaining part or all of special project Strategies after 
support ends. Volume VIII (Implementing and Sustaining Innovations) summarizes 



process of change at the local level. Thw/volume also contains a discussion 
of policy implications that are derived f rfcrm ^his study. 



the findings 
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Berne, Robert and L. Stlefei (1979) "Social Science Research and School Finance 
Policy: Documentable Impact or Wishful Thinking," American Behaviora l Scientist. 
November, 23, 207-36. \ fS=_ T L 

The purpose of this article Is to provide a basis for furthering olir under- 
standing of the impact of research on policy. The authors address th.s question 
of whether research impacts policy by formulating alternative ways to view this 
impact. This discusses five perspectives: 1) the problem-solving model; 2) the 
basic-applied model; 3} research as value clarification;. 4) research in the ser- 
vice of advocacy; and 5) researcher as educator. The authors elaborate on these 
perspectives and give illustrations of all five perspectives from thelarea of 
school financial policy. The final section assesses how these five perspectives 
can bring us closer to using social science research to answer the qJestion of 
the impact of research ^n policy. 



Bissell Joan S. (1979) "Use of Educational Evaluation and Policy Studies by the 

£ l 01C l ^ e ?7 -1 : tUr T* Educational Evaluation and P 0 1i.v in.LJ Vol 
no. J, p. 29-37, May-June. " 
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id concerning 
s to policy 
res s the 
should include 



The author discusses the characteristics of evaluative studies 
California State legislators. • A number of principles are identlf 
particular factors which tend to enhance the utility of evaluatio 
makers. These include: 1) policy oriented evaluations should 
specific issues of concern to the decision makers; 2) evaluatio 
attention to fundamental fiscal and legal questions in addition /to program 
effectiveness questions; 3) measures used in evaluations should/be straight 
forward and understandable to a wide audience; 4) evaluations Jnould aid deci- 
sion makers in developing a general understanding of educational practice; 
VaJH :*i u * t J 0M -hould b * r««Ponsive to the variety of values Zd program objec- 
ra^e ^"V^ offlcUlB - Xt Particularly important thml a bro.3 

range of outcome indicators be included in order to ensure attention to the 
diverse values held for educational programs. 



aiumberg, A. (1976) "OD's Future In Schools - Or is There 
Society . Vol. 8,, pp. 213-227. 



ae?" Education and Urban 



w / 
The author suggests that 0D in the schools today/ is a fragmented and isolated 
affair; at this time the 0D function remains nonaoeepted and noninstitutionalized. 
This situation exists* because of: (1) the concept of schools as organizations 
held by school administrators; (2) the nature and tradition of staff development 
services inyschool system organizations; (3) £bm paucity of data on the results 
of 0D programs in schools; and (4) the manner in which current economic conditions 
affect the setting of priorities and decision-making in the schools. 

* and is unlikely to change 



Boardman, Anthony E. and David Horowitz (1978) "The Potential of Social Science 
Research for Educational Management and Policy: Theory and Practice." Urban 
Education . Vol 12, No. 4, pp. 363-8^. 

This "tide presents some explanations for why social science research has 
been ot limited value *Co the* educational policy maker and makes some recommen- 
dations to both the policy analyst and the policy decision maker. 
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Bowles, B. Dean (1975) "Comments on the lannacoone Paper Education and Urban 
Society , Vol 7, No. 3, pp. 239-45. — — 

.Discusses four issues: The difference between policy research and basic 
research; the importance of policjfTseaearch for basic research; the. gener.al- 
izability of field studies; and the relationship of both of the professor in 
the professional school to the client in the field and of the professional 
school to the overall society. 



Boyd, W.L. (1978) "The Changing Politics' of Curriculum Policymaking for American 
Schools/ 1 Review of Educational Research , 48, 4; 577-628. 

To what extent should the public schools be agents to preserve our culture - 
and. to what extent agents to change it? To answer this question, it is desirable 
to] have a better understanding of the complex nature of current curriculum policy 
mailing, how this situation developed, and of the trends and implications of the 
changing politics surrounding this arena. x The purpose of this paper is to make a 
reconnaissance of this large, but minimally explored subject. Because of the 
scope of the topic * this paper emphasizes primarily the political problems 
crteated for curriculum policy makers by the simultaneous need for the school 
curriculum to maintain society while responding to pressures for societal change. 
Curriculum policy making usually proceeds quietly and incrementally rather than 
being character izedt by the model of rational decision making. As a consequence 
of the crises, a new shift is nonincremental or innovative curriculum policy 
making. Such more dynamic decision strategies are associated with what has been 
called "professionalization of reform." The nature of decision making also is 
affected by the nature of the structure of authority over curricular policy 
makings Finally, the paper discusses the constraints upon policy innovations 
that inhibit change. 



Bush, Robert N. (1976) Educational Research and Public Policy; Problems and 

Promise. Occasional Paper No. 12. SrWnrH ^ h >nd DcTclop 

ment in Teaching, Stanford University, CA. Presented at the ^ftnual Meeting 
of the American Educational Research Association (San Francisco, CA, April 20) 
xu p « 

V 

• The art of relating educational research to public policy is still primitive. 

Educational policy is formed mainly by tradition and the political pressure of 
• interest groups, while educational researchers study questions determined by 
the scientific community. Educational research has not noticeably influenced 
policy because trained researchers have been too few, resources too limited 
and efforts too fragmented. The need for illuminating educational policy by 
research is great, end the situation is becoming serious. The California 
Beginning Teacher Evaluation Study, which illustrates cooperation between. educa- 
tional policy makers and researchers includes these aspects: (1) The study is 
a genuine cooperative venture between the policy makers and researchers; (2) It 
has had sufficient funding from the several million dollars available to develop 
new programs; (3) The research and its eventual application requires time, which 
is available; (4) The study has been flexible in its planning and implementation 
stages; (5) -The policy makers and the researchers meet together continuously, 
and each teaches the other; (6) The project has had continued advice, to which 
li,tened ' fron • bro » d ■Pedtrum of persons on a research advisory board; 
17) The research end policy effort has been continuous; and (8) The project is 
taking advantage of spinoffs. If this study is a good example of research 
relevant to public policy, then educators are moving from a primitive state of 
organizing research and policy. 
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Charters, W.W., Jr. and R.J. Pellagrin (1973) "Barriers to the Innovation Process: 

Pour Case Studies of Differentiated Staff ing," Educational Administration Quarterly . 

9, 1:3-14. " ~ — — " 

Researchers at the Cent pi: for the Advanced Study of Educational Administration 
spent the 1970-71 school year conducting on-site observational studies in their 
first yepr of implementing Differentiated Staffing. The primary purpose^ of this 
study was to develop 4 better understanding of the process of organizational 
change in schools and of the problems) that commonly accompany such change. Based 
on their observations the authors hav4 Identified several major themes which affect 
the implementation of major educational change. r 

1. The fundamental but generally unacknowledged strain that exists, between the 
Ideology of teacher governance and the strategy of directed change. 

2. The gross unclarity in conceptualization .and definition of what the schools 
are attempting to implement through change projects. 

3. The heavy reliance on structural change (writing job descriptions, changing 
titles, altering organizational units) in the belief that appropriate beha- 
vior changes will automatically follow. v. 

4. The fallacious aasumption that a statement of general, abstract program 
values and objectives will easily be translated into new and appropriate 
behavior patterns at work. 

5. The. unrealistic time perspective of those responsible for educational 
innovation, according to which basic and far-reaching changes in instruc- 
tional roles and staff relationships are seen as accomplishable within a 

f year or two. ». 

6. The ambiguities and stresses that arise in the disjunction between the 
school district's established administrative structure and the temporary 
system for project management. 

7. The failure to recognize that teachers have scant training and experience 
in forming and implementing processes and procedures for collaborative 
decision making. 

8. The "conflict in goals, values, and interests, seen especially in the rela- 
tionships between the central office administrators, the project managers, 
and the school staffs (produced mainly by the requirements of their inher- 
ently different work contexts). 

9. The absence of managerial and monitoring procedures to .assure Implementa- 
tion and to alter pitas In the face ef contingencies that inevitably occur. 

10. The failure to recognize the severity of role overload among members of 
the Instructional staff when innovation is attempted. 

11. The tyranny of the time schedule In constraining change. 

12. The apparent assumption that schools/ need little additional resources (fi- 
nancial and personnel) to cope with the massive organizational disruptions 
during the periocj of transition from one educational program form to a 
new one. (p. 12) 

Chesler, M. A., Schmuck, R. and R. Lippitt (1963) "The Principal**. Role in 
Facilitating Innovation, * Theory into Practice . December, 2, pp. 269-277. 

1 

This article is a report on a pilot study that was designed to determine the 
influence of the principal's behavior on the development and sharing of innova- 
tive classroom practices. Improvements in teaching occur when teachers can 
share ideas and experiences, and staff interpersonal relations influence this*-"- 
sharing. The role of the principal in this process is both direct (encouraging 
an atmosphere supportive of experimentation and sharing) and indirect (his style 
of leadership may make teachers feel comfortable in discussing new ideas) . The 
authors found that the factors most relevant to teachers' initiating creative 
efforts were: 1) new practices could help solve problems important to them and 
their students; 2) a given practice is adaptable to their own teaching style, 
and therefore does hot require a great expenditure of time and energy; and 3) the 
school administration will support the new practice. 
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Cohen, David K. and Michael S. Caret (1975) "Reforming Educational Policy with 
Applied Research," Harvard Educational Review . Vol 45, No. 1, Feb., 17-43. 

There are several misleading assumptions that are held with regard to the 
relationship between research and policy. One assumption is that policies con- 
sist of discrete rfecisions: socisl research is expected to affect policy by 
influencing discrete choices among competing programs and methods. A second 
assumption is thst spplied resesrch is more suthoritstive than ordinary common 
sense ideas sbout policy: it is assumed thst sdvsnces in methodology will bring 
ebout corresponding sdvsnces in the authority and relevance of the information 
available for policy making decisions. The suthors investigate these assump- 
% £f 0M through the use of case study materials from educational policy research. 
The^ evidence suggests that resesrch sffects policy, but generelly it does so 
in odd snd unexpected" ways. The suthors attribute the unexpected conse- 
quences of researchVo the faulty assumptions, underlying the application of 
socisl research to policy. They offer alternative assumptions snd sn sltems- " 
tive conception of policy research which.argues thst "spplied resesrch is not s 
disinterested effort to improve policy, but rather a broad-aim social inhovstion 
designed to change the baais for decision-making." 



Coleman, Jjames S. (1978) "The Use of Socisl Science Resesrch in the Development f 
of Public Policy," Urban Review . Vol 10, No. 3, pp. 197-202. 

Although social policy makers snd researchers assume that research is s major 
part of the Information base upon which policy decisions sre made, social policy 
is by nature incompatible with the organizational structure snd time requirements 
for decision making in an administr stive system. Whereas socisl policy decisions 
sre timed in sccordsnce with legislstive sessions, social resesrch tends to be a 
lengthy process snd often does not provide results in time to be of use for policy 
making. There are, however, other less direct wsys in which socisl resesrch 
affects policy. One wsy in which policy can be used is to legitimate a challenge 
to existing policy. An example oi this type of use is the widespread citation ' 
of the 1966 U.S. Office of Education Report "Equality of Educational Opportunity" 
by civil, rights groups, plaintiffs in court esses, «nd in school bosrd dalibera- 
tions. Another wsy in which resesrch is useful to policy is in answering proce- 
dure! questions about how to increase the effectiveness of policies in force. 
In genersl, research on procedurel' questions will be more effective when it is 
done in-t£use, communicated, directly snd privstely to those in suthority, snd 
performed on s continuous monitoring basis. Resesrch ovf policy issues, on the 
other hsnd, should be done by an outside group which fo/uses on one psrticulsr 
issue snd disseminates resesrch results to all interestsd parties. The conclusion 
is thst socisl resesrch is less frequently used by an administrative body with 
suthority to legitimate a policy than by external groups, without suthority, to 
challenge existing policy. i 



Coleman, James S. (1976)" "Policy Decisions, Socisl Science, Information; and 
Education," Sociology of Education . Vol 49, No. 4, pp. 304-12. 

Examines the nsture snd accomplishments of social policy research and provide* 
general suggestions for Improving information flow to particular arias. The 
. suthor maintains that the majority of policy research hss been csrried out in 
educstion snd hss been less successful thsn it might be due to insufficient feed- 
beck systems. 



Coleman Peter (1979) "Teaching. A, Profession for Amateurs? Building Research/ 
Practice Linkages Via Informal Field-Based Training Activities for Teachers." 
Presented at the Canadian Educational Research Association Conference (Winnipeg, 
Manitoba, Canada, Sept. 24-25) 21 p. F 8 

Ways of implementing educational innovations produced by research into prac- 
tice in the schools are considered. It is stated that the most effective tool 
•for accomplishing this is inservice teacher education. The characteristics of 
an. exemplary inservice teacher education program are outlined and discussed. 
The suggestion is made that teacher centers are potentially the mbst important 
1£? k ..J 6 ** 661 * un iversity-produced research and changed practices in the schools, 
and that a consortium of university- teacher center-public school would be 
desirable. 

Corbett H. Dickson, III and WiUiam Firestone (1980) Constructing Caae S tudies 

^on Educational Change for Policy Makers . Presented at the Ethnography in 

^Education Research Forum (University of Pennsylvania, Philadelphia, PA, March), 

The popularity of using qualitative, or ethnographic, research methods to 
improve educational practices is increasing. However, an examination of how 
research findings are translated into policy is necessary to assess accurately 
the extent to which this goal can be realized. This paper assesses the effi- 
ciency of four strategies for increasing research use in poi icy making: 
enlightenment-transfer, engineering- transfer, enlightenment-transaction, and 
engineering- transaction. "Enlightenment" and "Engineering" are labels used to 
describe basic and applied research design. "Transfer" and "Transaction" 
describe methods of disseminating research information. It can be concluded 
that qualitative researchers interested in having research incorporated into 

. policy would be best advised to use either of the two transfer strategies and 
concentrate on improving the kind and form of information they present, 
guidelines for researchers for facilitating the incorporation of research into 

^ policy . are presented. An, example of an attempt to direct research to policy 
S prided Stratln8 addltlonal difficulties faced by qualitative rLearchSs, . 

\ ' : ' 

C0 ^n"\t r i 1972 l Tr**! 8168 f ° r 0r 8*nizational Innovation: An Empirical Compari- 
son, American Sociological Review . 37:441-454. «»f«m»x upmpari 

aocSwr^i 1 ^ 0 " 11068 aSpeCtS ° f Social intervention strategies which the 
sociological literature suggests will be conducive to change. Citing classical 

Ist^s of°both ISSS 1 r Ci °, l08y aUth ° r ^"hesizeHhat c£t2nH£2ir- 
«Ii:iV\ ?vJ nd J Vldual8 3X16 "Stations will be associated with change In 
order to test his hypotheses the author studied.ten Teacher Cor J Prn^f tI 

£S£ oTeducI^ 38 * ^« W^t^LSE ove^he 

quality of education in low income schools. The dependent variable is the number 

?nd^ Chn ° l08 , ie8 lntrodUced into ^e school through the program? ThSty-Sve 
R~tT«7 9 Wer ? f f Ct °^ ana ^ 2ed ' indicating seven factors associated wi'tn change 
Regression analysis, however indicates that only half of the variance in tne 

bi^of Infill! 19 T 1 ^; Alth ° U8h the aUthor notes tSt ^generaliza- 
m L flnd in83 is questionable, he tentatively concludes the following 3 

^H?n 7 M n inno y atl ° n 18 concelved a nd implemented is a product o?a 
combination of forces inside and outside the organization. 



(2) 

ie?leliZZ: ?S leaders of a profession and a 



a™^ l8ti J 8 ° f both ° ccu P at i°n and organization must be taken into 
fa f * °5 der 5° ^Piain innovation. 



Crandall, David P. (1977) "Training and Supporting .Linding Agents," In Nash, N. 
and J. Culbertson (eds.) Linking Processes in Educational Improvement: Concepts 
and Applications. Columbus\ Ohio: University Council for Educational Administra- 
. tlon. 



The purpose of this chapter is to present a view of the realities facing 
those playing linking roles and to give direction to renewed efforts to develop 
% training and support mechanisms for linking agents. The first section of the 
chapter presents the assumptions underlying the paper. The second consists of 
four principal subsections. The first presents three major perspectives on the 
current practice of linking agentry; the second addresses the nature of the 
resource system and the common sources and attributes of the Innovations that 
are typically the nucleus of, program improvement' efforts; the third discusses 
the client system as a complex social system and as an organization subject to 
a multitude of influences; and the fourth subsection deals with the host agency 
responsible for supporting the many needs of the linking agents they house. 
The third section of the chapter deals in depth with the linking agent proper. 
The many dimensions of this multifaceted creature are articulated and illustrated, 
the multiple roles to be played are described, attributes and skill clusters 
associated with the multiple roles are presented, and the question of selection 
versus ed ucatio n versus training is touched oh. 

_Some of the major underlying assumptions of this paper are that there 

tift 6 L f^M V k ° OWled8e U8er8; that they «*• di^e?ing "ses of ^ledg^ 

^ 6 a8ent C ° nCept i8 t0 be ^derstood andlefined with£ 
education as a macro-system, the uses of knowledge and the conditions affectine 
• knowledge utilization need clarification. Toward this end, £ve major uses 22 

ions'ar^tinf EE - ? - <T Mt f ° rth ' as have "een IZl oftne^" 
knZSlJfff™ ? k ?° Wledge use * ™ e and their sub-uses are to create new 

knowledge-conclusion research and policy research; to develop new svStheses- 

Z!^ ri ! nted / ynthe8eS and Practice-related synthesis; to atL^n^ devel- 

" Ct deVe J?P ment lde * development; to achieve new o> WoveT 
practice-improved policies and programs, and newly instituted policWoT 

HallT 3 ' *£< t? ? Pr ° Ve trainin8 P"grams-in-service programs and pre-service 
2£X.t2£ 18 f ™ Perspective of Suca^nal 



Davis Junius (1976) "Potentials of the NLS Data Base for Issue-Oriented Analyses 
and For Influencing Educational Policy Decisions." Presented at Se AanSal 
Xpt^-^.'J 6 p merlCan Ps y chol °8 ic al Association. (84th, ^WashUJon^, 

The potential of research which uses_ the data base from the National 

£2S5?i!™ f 0rmulated and ln toowln 8 how to report research sTthat It is 
K J ly , relevant ' A "P°" Prepared by the Rand Corporation sees the M.S 
data as having potential f for four stages of policy formulation- m Earlv 
warning of forthcoming problems, (2) Issue exploration^ ( 3? idlnMf l< 7 c 
specific problems, and (4) Evaluation of th^SS^f St^SS^cJSaS 

8U8 f 8 J 8 0nL7 tW ° 8tage8! ^ or confimatJon of a 

P-£^ 



Deal, Terrence (1979) Linkage and Information Use In Education Organizations . 
Presented at the Annual Meeting of the American Educational Research Associa- 
tion (San Francisco, CA, April) 14 p. 

Linking activities between knowledge production and actual practices in 
schools need to be guided by understanding, of how educational organizations 
work. Educational- organizations evidence unique organizational patterns, in- 
cluding diffuse goals, weak connections between activities and outcomes, 
political vulnerability, and working conditions that emphasize individual 
autonomy. As new knowledge is linked to schools and used by administrators 
and teachers, these organizational patterns create Important opportunities and 
...... Pitfalls. In schools, knowledge use and transformation may serve purposes 

other than the intended objective of change or reform. Knowledge utilization 
also provides legitimacy to existing programs, new enthusiasm, new beliefs, 
and an improved public Image for the schools. New knowledge may also v undermine 
existing relationships within schools or between schools and the outsifce en- 
vironment. For instance, when knowledge is used to redefine roles or relation- 
ships, change practices, or alter characteristics of schools, conflicts arise, 
without >*n adequate understanding of organizational realities in schools, 
efforts to increase the role of knowledge in change and reform will produce 
more problems than solutions. 



Derr, C.B. (1976) "'pD' Won't Work' in Schools," Education and Urban So ciety. 8, 2: 
227-241. : : *" 

The author claims that OD has failed to have far-reaching' Impact because, like 
other curricula, it was packaged for teachers and administrators such that it 
became a program and a fad rather than a continuous process. In addition, various 
dimensions of the traditional school organization do not lend themselves- to the 
current form of OP. The dimensions are; (1) the lack of common indicators of 
performance; (2) the nature of the school environment; (3) the high autonomy heeds 
of people who work in schools; (4) the low degrees of required interdependence; 
(5) the civil servant mentality; and (6) the shortage of economic resources. 



DiC Ed^M' b ^ a p (1 ? 80) :; U i J iliZati0n ° f Valuations of Small Scale Innovative 

Educational Projects," Educational Evaluation and P olicy Analysis . Vol 2, No. 6. 

This study focuses on the evaluations of Title IV-C projects in the state of 
Minnesota funded in FY 1975 and FY 1976. The purpose of tL study was It eL- 
m u£? f S evaluation utilization by addressing the following questions: 

7 £?l J 8 reported level of utilization of the evaluations examined?; 
(2) What factors are related to the reported level of utilization?; (3) How is 
the level of utilization related to the role of evaluation in the process of 
planned change? In addition, the author reviews th/literature <on "underutili- 
liHtH^i categorizes the arguments into three broad categories: (1) Inherent 
organizational inertia causes resistance to any change; (2) The quality of eval- 

™ a ^ a nee f. ^r ement; the 8tate ° f the art 18 in lts ^fancy and better 
methods and techniques must be developed; (3) the "two communities" notion: 
differences in experience and outlook between decisionmakers and evaluators make 

dSL S 10 ? ? iffi , CUlt - IC 18 fr0m Che8e broad categories that the author 
derives the Independent variables for the study. The findings suggest that Jihese 
evaluations have had an Impact, not only at the project level, but also in 
helping to validate exemplary projects for wider dissemination. Based on this 
study, the author recommends that evaluators should adopt a more collaborative 
role, involving the decision makey/and the staff in decisions about the evalu- 
at ion. 



Doyle, Walter and George Ponder (1978) "The Practicality' Ethic in Teacher Decision- 
Making," Interchange . 8, 3:1-12. 

This article focuses on the decision making processes which appear to underlie 
teacher reaction to change proposals. The authors assert that the practicality 1 
ethic is a key link in the knowledge utilization chain in schools. The practi- 
cality ethic refers to the evaluative process which takes place when a teacher * 
makes the initial decision regarding the implementation of a proposed change in 
classroom procedure. Proposed changes which are seen as practical will be incor- 
porated, at least tentatively into teacher plans. This paper, thei, is a study 
of the perceived attributes of innovations and the way in which these perceptions 
determine the extent to which teachers will evaluate and attempt to incorporate 
proposed changes. The authors contend that decisions about practicality result 
from the complex interaction of several variables. They further assert that 
teachers appear to use three general criteria to determine if an innovation quali- 
fies as practical. These criteria are instrumentality, congruence and cost. 
Instrumentality means that an innovation or change proposal must describe a proce- 
u!!u depicts classroom contingencies. Congruence refers to the extent to 

which a proposed innovation is consistent with teachers' perceptions of their own 
situations. Cost, in this context, refers to the ease with which the proposed 
innovation can be implemented and the potential rewards for adopting an innovation. 

Everhart, Robert B. (1976) "Ethnography and Educational Policy; Love and Marriage 
or Strange Bedfellows?" Anthropology and Education Quarterly . Vol 7, No. 3, 
pp. 17-25. 4 

Reviews some major characteristics of ethnography as they apply to policy 
issues and policy formulation, and ethnography's utility from the perspective 
of^ how public educational policy is made. Discusses the potential contributions 
of ethnography and the responsibilities ethnographers must consider if their 
work is to have an impact on public policy. 



Florio, David H. (1978) "Research and the Politics of Education," Educational 
Forum , 42, 4, 490,-501. " 

We are in th^s midst of a turning point in federal education policy. The 
research community has a major contribution to play in the shifting pattern of 
federal education support. The author suggests that several assumptions must 
be understood about the use of research in policy making: (1) First, it should 
be understood that social science evidence will be used by policy makers for a 
variety of purposes (e.g., confirming past policy decisions, challenging posi- 
tions of cithers, making a case for or against popular ideas, etc.); (2) It 
should be clearly recognized that politics is the business of allocating Values 
and scarce resources. Social science evidence, therefore, is mixed with value 
choices in the development of public policy; (3) Research often provides addi- 
tional, therefore more complex, perspectives on social problems and, in so 
doing, challenges simplistic or inadequate solutions; (4) policy jjakers should 
understand the diversity of inquiry that fits under the^bric orresearch. 
Policy makers, participants in the R&D community, and users of R & D resuljstf" 
must take an increasingly sophisticated view of the research enterprise. Jijhe 
benefits and limitations associated with immediate, short term, and long term 
investments must be articulated. If these opportunities are missed, federal 
support for research will surely decline or, worse, will be limited to immedi- 
ate and short term inquiries. 



5b 



Florio, David, Michael M. Behrmann and Diane L. Goltz (1979) "What do Poliev 
. makers . Think of Educational Re 8 earch and Evaluation Or do xJeJ?" Eduction*! 
Evaluation a nd Policy An a1 y «»« vr N . 6> 6l . 87> ° r fl0 they? Educational 

wlT^T'Sl con 8 re88lon al "aff members dealing with educational legislation 

a£ vSu^ e of d educ"^!- ° f **«-.tW«d the relative importance 

SnnTnir S * ducat ^ 0nal inquiry in various stages of the policy process. The 
lt«tlT th ? 3tUd * " Veal much ab0Ut the disparities beLeen tSe educational 

4 n ™ e scuay are. 1; There is a relatively small arouo of staff in 

- aTcon^s 8 ^; * Principal responsibility for educatiLrpolfcHecLionlf 
resSrS LS? ? ? \ 8reater ^standing of the distinctions among types of 
social r es ear h !i i- P ° licy re8 « a "h and evaluations as oJpoLd IT 

social research or data collection; 3) There continues to be a stroni belief 

Sd\T"har w'ff th V7 laf human ^ P^blems cTbe rans- 
4) K researchers wish to°h technical ' •rganizational or programmatic solutions; 
; researchers wish to have an immediate. impact on leaislative decisions, 

!u!L?^» * fairly ^ rr0W range of choice; 5) -Addressing policy maker's 

questions does, however, enhance a researcher's credibility and opportunity to 
introduce additional questions pr challenge popular assumptions; 6) The initial 
burden fpr improving the use ef knowledge will fall more heavily with the re- 
searchers; 7) There is currently enough valued use of 'educational research to 

per " ption that f uch ln 1 ulr y can play an important role in the im- 
provement of educational policy. 

Fox, R.S. (1972) "Innovation in Curriculum: An Overview," Interchange . 3, 2-3. 

The author views innovation in curriculum as gaining its thrust from a new 
set of value priorities - humanism, concern for racial and socioeconomic minor- 
ities, support of pluralism and diversity, increased interest in affective 
development, education for the very young, and legitimization of the search for 
value. These priorities are coupled with new understandings of how lekrning 
takes place, which east the- learner in an active, "responsible, self-acRalizing ■ 
role and, at the same time, support the development of precise, highly organized 
sets of learning materials designed to achieve clearly defined -objectives as 
efficiently as possible. A literature review covering each of the aforementioned 
concepts is provided, and implications of these concepts are examined in relation 
to curriculum design, instructional materials, professional roles, and the need 
for in-service education, 
(author abstract-modified) 



Fullan, M. , Miles, M. , and G. Taylor (1980) "OD in Schools - The State of the Art 
pP^ U 121-183 ReVieW ° f Re8earGh in ° D '" Sgggg °f Educational Research . 50, Spring, 

■ ' I 

Organization Development (OD) is a change strategy for organizational self 
•development and renewal. Adapted f rem business settings, it has been used in 

8 n5°°i! r e I.« e Pa8t 15 yea "- T^"^" widely different images of what OD is, 
and widely different claims made for its value or worthiessness. The field of 
OD in education is badly in need of stock taking. In this review the authors 
assess ttie state <^f the art of OD in four respects: (1) critiquing and clarify- 
%T]^ Bt A° f 18 ' a ? d a88um P tlon « of OD in general and as applied to educa- 
S'n L 81 y ? a ? d *™ lyzia * the varlou « models and operating characteris- 
tics of OD in practice (conditions and strategies affecting its initiation, 
implementation^ and continuation); (3) assessing the impact or outcomes of OD on 
achievement, productivity, and attitudes; and (4) reconsidering OD's future, and 
suggesting policy implications for educational agencies at different levels 



1 Fullan, M. and A. Pomfret (1977) "Review of Research on Curriculum and Instruction 
Implementation," Review of Educational Research . 47, Spring, pp. 335-397. , 

The authors define Implementation, which is claimed to be separate from plan- 
ning and adoption processes, as "the actual use of an innovation or what an 
innovation consists of in practice." It is suggested that there are at least 
five dimensions of implementation in practice — changes in materials, structure, 
role/behavior, knowledge and understanding, and value internalization. Approxi- 
mately 20 previous research studies were -examined by the authors in relation to 
how to measure Implementation and to the determinants of Implementation. Several 
factors which have, a significant Impact on the Implementation of innovations are 
identified for each of four broad categories; (1) characteristics of the innova- 
t "<m; (2) strategies; (3) characteristics of the adopting unit; and (4) character- 
istics of macro-sociopolitical units. 

Fullan, M. (1972) "Overview of the Innovative Process and the User," Interchange . 
Vol. J, 2-3. 

The focus of this overview is on the role of the user (student, teacher , 
parent) in the educational change process. The first section reviews literature 
by critically examining the assumptions of existing models and approaches to edu- 
. cational change, and by analyzing evidence on actual user experience with educa- 
tional innovations. The main conclusion is that the model process of change 
whereby innovations are developed external to schools and then transmitted to 
them has led to no significant change at -the user level. 

In the second section the analysis is extended by relating the basic conten- 
tions afcd findings of the authors in this Interchange volume to the criteria 
for effective change identified in the first section. 

The overall conclusion in the final section of " this overview is-^hat a radi- 
cal restructuring of the role of the user and a complete reversal of the direc- 
tion of influence in the process of change are required if effective Innovations 
are to occur. Finally, the main prinicples and elements of reform - techniques 
activities, organizational forms - necessary to support this active user role, 
and other factors beyond the user level that might Impinge on this role, are 
outlined. 

(author abstract, modified) 



Fuller, F.F. (1969) "Concerns of Teachers; A Developmental Conceptualization," 
American Educational Research Journal . 6, 2:207-226. 

From the author's own studies and from the data of other investigators, three 
phases of concerns among student teachers and teachers were identified. In the 
pre-teaching phase, students in education courses tended to have no major concerns 
about teaching. In the early-teaching phase, student teachers were most concerned 
about class control and their own adequacy in the classroom. In the late-teaching 
phase, experienced teachers tend to focus on pupil gain and self evaluation. 

i . 



ERIC 



6u 



Giac<juinta, J.B. (1975) "Status Risk-Taking: A Central Issue in the Initiation and 
Implementation of Public School Innovations," Journal of Research and Development 
in Education . 9, 1. / ~ 

This article examines conceptual and empirical work bearing on risk-taking and 
on public achooj.* as complex organizations, in an attempt to better understand why 
planned change in- schools has been, and continues to be, difficult to effect. 

The paper focuses on schools as complex organizations consisting of a set of 
formally defined positions or statuses. Associated with each status is a set of . 
expectations for behavior, or roles. An educational innovation means the modifi- 
cation of patterns of role interaction - leading to status risk-taking. 

The author reviews literature on risk-taking, and discusses the relationship 
between the uncertainty which results from status risk-taking situations and the 
resistance to organizational innovation. 

The author concludes by suggesting that participation may act to extinguish 
uncertainty or to suppress risk-estimation, and thus facilitate effective change. 

Giacquinta, J.B. (1973) "The Process of Organizational Change in Schools," in Review 
of Research in Education, (ed.) F.B. Kerlinger. Itasca* ILL: FE Peacock. 

This review of literature is basically atheoretical in nature and provides the 
basis for two tentative propositions about organizational change, foe first is 
that the extent of change in any school r s organization and speed with which it 
occurs depend upon multiple factors: the nature of the innovation introduced, * 
the tactics used to introduce it, the characteristics of the individual school 
members who must carry it out, and the properties of the school structure in which 
it is introduced. The second proposition is that an attempt to change a school 
Organizationally, when successful, proceeds in three basic stages: initiation of 
the innovation, implementation, and incorporation as a stable part of the organi- 
zational atructure. The factors of change set forth above and the three basic 
stages in the procesa are discussed throughout the paper. Finally, a theoretical* 
framework for the future study of change emerging from this analysis is proposed, 
and procedures for collecting and analyzing data that will be required for future . 
exploration of change in schools, as well as in other complex organizations, ate 
suggested. 

^fUST/. Hendrik D - (1978) " A ^el for Educational. Research and Development: 
1985, Occasional Paper No. 44." Presented at the National Center for Research 
in Vocational Education Staff Development Seminar (Columbus, OH, June), 26 p. 

Three broad statements summarizing nine axioms underlie a conceptualization . 
of the character of behavioral and social research: (1) Social and behavioral 
science is inherently reflexive, (2) Ethics and epistemology are inseparable, 
and (3) The inherent complexity of behavioral and social phenomena must be 
reflected in the models developed to guide the support of productive educational 
research and development (R&D) . The idea of differing epistemologies suggests 
that ignorance of or failure to recognize the worth of others* work leads to 
conflict between practitioners and- researchers, research administrators arid 
politicians as well as to slow adoption of research. A promising interaction • 
model fdr educational research focuses on the outcomes of research and those 
who have a stake in the outcomes. Relationships between "stakeholders" and 
outcomes vary. The form of an outcome is related to accessibility to stake- 
holders; stakeholders may have several relations in respect to a research 
outcome-primary, secondary, or tertiary. Policy Implications flowing from the 
model lie in three main areas: the legitimization of a broader array of R&D 
activities, functions, and actors; decentralization of authority and respon- 
sibility for educational R&D; and the demand for much more careful analysis and 
attention to policy in its qwn right. 

6i 



Granville, Arthur C, et.al. (1978) The Impact of Evaluation: Leasona Drawn 
from the Evaluations of Five Earl/oT t ldhood Education Programs . Printed 
at the Annual Meeting of the American Educational Research" Association (62nd, 
Toronto, Ontario, Canada, ManffiN 27-31) , 45 p. 4 ' 

Five different program evaluations were described to Indicate those ' 
qualities which make an evaluation effective or not effective. Evaluation 
effectiveness was defined as impact on decision making or long-term policy 
formation, and influence upon a variety of audiences, Robert D. Matz described 
the first chance project, and concluded that the evaluation methodology used 
to inform policymakers should be distinct from the approach used to improve 
teaching. John M. Love, who was associated with the National Home Start Demon- 
stration Program, felt that several factors contributed to the value and use of 
evaluation information: Evaluations planned with the program; timely reports; 
rigorous experimental design; reapecp between -agency and evaluators; demon- 
strated relationships between process and outcomes; and non-controversial appeal 
ot the program. Project development continuity was discussed by Arthur C. 
Granville. Factors affecting evaluation impact included the relevance of quan- 
titative data; sociopolitical acceptability of the implications; and pertinence 
to evaluation criteria. Allen G. Smith, who discussed project' follow through, 
supported close relationships between evaluation research and curriculum. 
Lawrence J. Schwelnhart of-the Ypsilanti (Michigan) Perry Preschool Project 
recommended intensive one-site studies, determination of the feasibility of 
longitudinal studies, and adequate funding. 

GUb il^ n 9?*5! 1 25! 3 8." The FallU " ° f Educ * tlon * 1 Evaluation," Educational Tech- 

to I!!!.,!*?^ 1 ? 11 * 1 ! et !? 0 ? 8 ° f ev * lu * tlon h * ve educators in their attempts 

nL!T!?!2 J i mPaC 5 ° f innov * tlona in operating systems. Innovations have 
• persisted in education not becuase of supporting evidence but despite it. One 

5*51?^ J! ! Hi8her , Horizon8 P r °8™» la New York City where test data failed 
to affirm what supervisors, teachers and clients insisted was true - that the 
• program was making a difference. Another example is the Coleman report which 
- states that schools bring little influence to bear on a child's acSevemenl 
that is not explained by his background and general social context. Although 

il£+ g T Vm ll y ! CCeP i! d that thi8 18 n0t true ' «™l"«ive techniques fail to 

\ ™" aUth ° r 8u 88 e8ts that the world of evaluation requires 
til ^^K ' The / Uth0r ll8t8 « d dosses six sign, that indicate that evaluation 
has not been effective: Avoidance; anxiety, immobilization; vague guidelines; 
ISS^"? 8 ^ n f lflc8nt fences. He then discusses what evaluation has 
lacked. 1) Lack of adequate definition of evaluation. Evaluation cm be de- 

3) n ^ll n o Tll *"T k 17 " bi " 8r y W8V -J 2 > ot adequate evaluation theory; 

dL£d kSLST 8 " ab0U J deci8ion Processes. Programs to improve education 
depend heavily upon a variety of decisions and a variety of information is 
needed to make and support those decisions; 4) Lack of criteria. The mere col- 
lection of data does not constitute evaluation - there is always at least a 

««t^!^^ 8 T JU ? 8 T nt8 ab °v t the dat * ln tem8 of or explicit value 

structure; 5) Lack of approaches differentiated by levels. The problem of 

;^!m^* 8 i ^V?™/! U * ed here ' 8t «»* f"» the fact that the evaluator's 
traditional point of focus has been microscopic (e.g., the individual student, 
the classroom, or the school building) rather than macroscopic (e.g., the 
school district, the state system, or the national network); 6) Lack of 
mechanisms for organizing, processing and reporting evaluative information; 
7) Lack of trained personnel. 
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Guba, Egon G. and David L. Clark (1974) The Configurational Perspective: A New 
View of Educational Knowledge Production and Utilization . w M hin.rnn. n r. . 
Council for Educational Development and Research, Inc. 

ThlsVaper first belefly reviews and analyzes the historical development of 
Federal level Involvement In educational knowledge production and utilization. 
This analysis concentrates on the modern era of Federal concern for KPU (1974- 
74) . The views of educational KPU that have emerged during this period are 
Identified and evaluated. The authors concentrate on the systems view which is 
the conceptual structure currently guiding educational KPU policy. Finally the 
authors propose an alternative conceptual view, the conf igurational view. This 
new perspective is compared and contrasted with the systems view on the dimen- 
sions of completeness, realism and balance. The authors conclude that the use 
of the configurational model to generate Federal policies and programs for edu- 
cational KPU would result in more productive relationships both/between Federal 
funding agencies and KPU units in the field and among the latter units, by 
building upon what has been developed by application of systems view and by re- 
fining and extending those achievements. Some illustrations of how adoption of 
the configurational view might affect Federal level policy and programs are then 
offered. 

^or^L™? S ^ T Louck ? <i 981 > " p «8ram Definition and Adaptation: Implications 
' for In »«v*ca. Journal of Research and D^l ppment in MuLm- . 14, 2:46^58? 

.^ U8 " f ?i in8ervice training requires that inservice planners understand define 

5« IJJ!? V * doinl " tr * to «. «aff developers and evaluator. have found useful 
inn0V • ti0n, and th * ir • d *P t "ion. involve, identifying anTdafSlS 
the operational component, of an innovation, then analyzing the Mturn Tof 

fSur^r'VLST^; ^ " fer <° thi/conlept ^^jL^og 
figurations . A conf iguration refers to the form a process takes on during use 

f °? "J"! C ° *° y pr ° 8rim whlch "« ulr * 8 Sh«g« ^ behavior of the 
individual, involved Defining configuration, of innovation. anS m^nitor^ their 

Zd" a l ln ' ervi " intent can be t.rg.tedlore^w.rf' 

« Ml h.l in8erv J c f can be «qu«nced and phased to ensure a systematic 
rational change process; (3) enables development of dear procedures for aonitor- 

cUrSy Pl ": n : o *.iro? f s th h e lrmOVafei0n in the Cla " rOOOS (4) thelo^p^c^eip 
thTnew pX^ 1 mel%p^Lnany e8y * reqUirin8 °* *»« use Sf 

Hall, G.E., Trucks, S.F., Rutherford, W.L. and B.W, Newlove (1975) "Levels of Use 
of the Innovation; A Framework for Analyzing Innovation Adoption/ 1 Journal of 
Teacher Education , 26, l;52-56. ■ 

In this article the authors describe a hypothesized dimension of Innovation 
adoption which attempts to Assess what the Individual Innovation user actually 
does In using an Innovation. I^e authors assert that Innovation adoption la 
not a decision point but a process which each Innovation user experiences Indivi- 
dually. Therefore, members of a user system (I.e., teachers, principals) demon- 
strate a wide variation in the type and the degree of their use of an Innovation. 
In order to account for Individual vat la t Ion In the use of an Innovation, the 
authors have proposed the Levels of Use of the Innovation concept. The Level of 
Use (LoU) dimension describes the various behaviors of the innovation user through 
8 stages which range from spending nu^t efforts in orienting | to managing and 
finally to integrating use of the innovation. The purpose ol the LoU dimension 
is, to describe behaviors of innovation users and does not focus at all on attitu- 
dlnal, motivational or other cognitive aspects of the user. The authors discuss 
several Implications of the LoU dimension. 
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Havelock, R.G. (1973) The Change Agent's Guide to Innovation in Education , Englewood 
^filiffs, NJ: Educational, Technology Publications. 

This guide to the process of innovation in education was wr*£teil for educators 
working for change at all lev*ls~of the educational system. Lt discusses now 
successful innoyation takes place and how change agents can mjLage their work 
toward this end. The author presents a six stage change modelV-— 

1. Relationship - the successful change agent must first develop a viable 
relationship with the client system. 

2. Diagnosis - after establishing an appropriate relationship, the change 
agent must guide thf client in making a diagnosis. 

3. Acquiring relevant resources - seven major purposes of resource acquisition 
are given under the acronym DAETEJM: Diagnosis, Awareness, Evaluation- 
before-trial, Trial, Evaluation-after-trial, Installation, Maintenance. 

4. j^opsing the solution - the author examines four steps in this stage:' (a) 
deriving Implications from research; (b) generating a range of solution 
ideas; (c) feasibility testing; and (d) adaptation. 

5. Gkining acceptance - after a solution has been developed and adopted by 
the change agents, it must be presented to the client system. 

6. Stabilizing "the innovation and generating self renewal - as the last stage 
in the fchange process, the client needs to develop an internal capability 
to maintain the innovation. 

Havelock, R.G. (1968) "Dissemination and Translation Roles," ip knowledge Production 
and Utili zation in Educational Administration , (eds.) T.L. Eidell and J.M. Kitchel. 
Eugene, OR: Center for the Advanced Study of Educational Administration, Univer- 
sity of Oregon: 64-119. r 

■ % 

' v 

The purpose of this paper is to analyze the roles, institutional arrangements 
and problems involved in the process of linking research information to potential 
users. There are a number of roles which are linking roles in one way or another. 
The author identifies eight major types of linker roles: conveyor, consultant, 
leader, innovator, defender, knowledge-builder, practitioner, and user. The 
resource syste# includes the scholar, the researcher, the expert, the producer 
and packager and applied R&D. The consultant and the conveyor link this system 
to the client system, which includes the defender, tlie innovator, the leader and 
the user. The linker's activities pan be grouped into three kinds of processes: 
getting infQtrmation (input), processing information (throughput), and distributing 
information (output). According to the author, there are four things that have 
to be done in order to build a functioning system of knowledge linkers: (1) build 
an institution which includes and supports the- required roles (installation); (2) 
recruit candidates t6 serve in these roles (recruitment); (3) train recruits to 
fill the rolto (training); and (4) supply the equipment necessary to do a good 
job (equipping): 

Hayman, John (1979) The Case for Internal Linkage: Functional integration 
within thg Organization. Presented at the Annual Meeting of the American 
Research Association (San Francisco, CA., April' 8-12) 32 p. 

This paper discusses the increasing attention given recently to change 
theory and knowledge utilization in education and to the development of dis- 
semination mechanisms in educational systems. Also discussed are the concept 
of "Linkage," a process of connecting users with resources in an efficacious 
way, and the role of linkage agent, a person who helps to bring this about. 
The paper argues that internal linkage agents are more effective in educational 
systems than external agents, and it attempts to clarify the theoretical and 
the operational meaning of the term "linkage" as it is being used in educational 
dissemination and diffusion activities. The paper concludes that knowledge 
utilization is more than a matter of convenience or desirability. It is a 
critical matter that relates directly to an organization^ survival and its 
ability to achieve its goals. 



Hood, Paul, et^al. (1976) The Educati onal Information Market Study. Study o f 

Information Requirements In Education. v*t u».r t.k ,a El5nal Research 

and Development, San Francisco, CA. System Development Corporation, Santa 
Monica, CA. National Institute of Education (DHEtf) , Washington, D.C. 

Data on the characteristics, Information needs,, and lnformatl6n dissemina- 
tion patterns of actual and potential users of educational Information were 
gathered through field Interviews with persons representing a variety of 

educational roles and geographical locations (Vol. I) as well as through 
a nationwide mall questionnaire survey (Vol. II) questions for the study were 
generated from an educational Information use model which hypothesized rela- 
tionships between user characteristics, infortnafton needs, and Information 
sources employed. Results indicate that while there are many differences 
among respondents for the variables analyzed, there are distinct patterns of 
Information use related to personal style and type of educational position 
held. This study forms half of a larger study intended to provide guidance 
for decision malting and planning at all levels in the United States Education 
Information Service Complex: Appended are the mail survey sampling design 
and the mail survey instrument. 

Hood, Paul D. and Carolyn S. Cates (19 78) Alternative Approaches to Analyzing 
Educational Dissemination a nd Litikaaa Roles and Functions. Educational 
Knowledge Dissemina tion and Utilization . (Occasional Paper Series) , Par 
Vest Lab for Educational Research and Development, San Francisco, CA, 109 p.* 

The objective of this paper is to consider alternative frameworks for the 
description and analysis of linking agents' roles and functions and to explpre 
the^ implications of the frameworks for linkage, training and support. Section 
1 draws attention to the variety of terms and contexts used in the literature 
on educational knowledge utilization and draws distinctions between three 
widely used terms: Knowledge Utilization, Knowledge Transfer, and Planned 
Change. In Section 2, general orientations toward change are examined from 
a variety of perspectives. Ip Section 3, seven widely quoted "systematic" 
models of change are reviewM in terms of their Implications for linking 
agents, macrosystem linkages,' linking roles, and temporary and permanent link- 
ing systems. In Sections 4 and 5, recent conceptions of linkage and linking 
agents are examined and comparisons are drawn among the linking agent roles 
described in the literature. In Section 6, directions for future development 
are suggested. The Implications of previous conceptualizations and the sug- 
gested future directions for linking agent training and support are discussed 
In Section 7. 



House, E.R. (1979) "Technology vs. Craft: A Ten Year Perspective on Innovation," 
Journal of Curriculum Studies , 11, 1:1-15. 

The author sees educational innovation as the systematic attempt to change 
schools through introducing new -ideas and techniques. Technological, political, 
and cultural perspectives have dominated thought on innovation over the last ten 
years. The author notes the growing tendency to interpret specific innovations 
within the context of larger social changes. 
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House, E.R. (1974) The Politics of Educational Innovation . Berkeley, CA: McCutchan. 

The author describes two attempts in Illinois to introduce instructional inno- 
vations - programs for the gifted and the PLATO computer assisted instruction 
program in several colleges. Various aspects of it^wrvatipn are discussed, 
including, where it originates, how it spreads among practitioners, and what its 
consequences are for the interna}, workings of the school. Internal political 
and social structures are emphasized as well as how the school relates to larger 
social and institutional networks. The author believes that face-to-face personal 
contacts is a primary factor in the occurence and frequency of innovation. 
Furthermore, an advocate working within the system is required to overcome the 
rigid political and economic structures within a school if innovation is, to suc- 
ceed. Rather than the usual top down pattern of innovation diffusion, the author 
suggests greater rewards for and promotion of initiatives at the bottom. 

House, E.R., et al. (1972) "A Tjest of the Research and Development Model of Change," 
Educational Administration Quarterly , 8. 1 (Winter) ; 1-14. 

This study questions the "research and development" model, which has been the 
basic educational change strategy employed by the Office of Education and other 
top policy groups for the last several years. The Implication is that educational 
change is more dependent on the internal workings of the school district than on 
feeding carefully engineered products to practitioners. 

The authors suggest that the "social interaction" change model, which sees 
change as the result of the socipl relations network within the adopting unit, is 
more appropriate than the R&D model. 

***** 

House, E.R. (1971) "A Critique of Linear Change Models in Education," Educational 
Technology , 11, 10. 

Tha^au^hor argues that the Clark-Guba model of educational change - with its 
four distinct functions of research, development, diffusion, and adoption - is not 
very successful. A primary reason is that the model incorrectly assumes that 
actors at various stages are pursuing similar goals or are held accountable to a 
common goal by some force. The general belief in this model may reflect the bias 
of people's desire for rationality and order. 

Hull, William L. , et.al . (1978) '.'Research Needs in Diffusion, Change and Informa- 
tion Systems." Presented at an Invitational Conference on Research on Dissem- 
ination and Utilization in Education at the University of Texas. 

Through an examination of the background and current status of programs at 
the National Center for Reserach irv Vocational Education, one can identify many 
research problems and issues. Some of these current National Center Programs 
include a diffusion program (combining five U.S. Office of Education-sponsored 
projects) Research and Development (R&D) efforts (focusing on operational prob- 
lems in National Center Diffusion and Information Systems) and information 
system activities (including the Eric Clearinghouse on Adult, Career, and Voca- 
tional Education, the National Center Clearinghouse, and the Resource and 
Referral Service) based on these and past activities, eight issues can be iden- 
tified as potential research topics concerning educational dissemination and 
utilization: 1) A discrepancy exists between national priorities and local user 
needs for the dissemination of R&D results; 2) Communication processes need 
further development; 3) Practitioner capability to obtain and use R&D information 
needs strengthening; 4) Diffusion of innovation^atrategies have not been vali- 
dated; S) Information about exemplary programs is not valid or reliable; 6) 
Methods of conceptualizing and measuring the impact of R&D based information 
have not been developed; 7) The issue of selective versus comprehensive informa- 
tion systems needs resolution; and 8) The roles and relationships of public and 
private agencies active in dissemination and utilization need clarification. 
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Hurst, Barbara Martin (1979) Ordarlng Theory: A Methodology For Sequencing Stage s 
of the Adoption Process. Presented at the Annual Meeting of the American Educa- ' 
tional Research Association (San Francisco, CA, April 11) 26 p. 

This paper describes a method of research* to examine the process of effecting 
educational change through adopting innovative .ideas. Twenty-nine elementary 
tearchars in the process of adopting a complex curricular innovation were studied. 
Because ofi> the complex nature of this process, ordering theory was used as the 
method for^analysing the adoption stages. Ordering, a method of defining both 
linear and non-linear sequencing, measures both sequential skill -development as 
well as the degree to which positive attitudes are necessary in the adoption pro- 
cess. Results indicate that adoption involves four levels: Formation of a positive 
attitude toward the innovation, acquisition of general knowledge and concepts, a 
combination of further knowledge and increasing positive attitudes, and refine- 
ment of knowledge and use of the* innovation. The use of ordering theory was 
deemed appropriate in that it provides a more thorough analysis of the research 
data. Utilization of this research methodology should change the nature of 
research efforts and the way in which innovations are introduced into school 
, systems. ^ 

Iannacoone, Lawrence (1976) Using Research in School Reform , 8 p. 

This paper suggests that there is an underlying political revolution in 
educational governance. Furthermore, th^re is a need for concern with the 
changes needed in the roles of researchers and state policy le^vel policy makers 
in order for research to play a significant part in helping shape reform in 
education. Given the present political conditions there are unusual dangers i 
for researchers trying to serve state policy makers. The initiative for 
educational changes has shifted from t^e local district to the national govern- 
ment and subsequently to state leadership* The legislative patterns of recent 
years have increased further the separation between responsibility and opera- 
tion. Increased centralization at the state level is combined with the creation 
of multiple new points of access in educational governments ranging from teacher 
negotiations to site level parent advisory committees. This is the context lL 
within which the state policy maker is asking the educational researcher to 
provide data for his predetermined solutions . The researcher is asked to make 
antithetical policies work despite their inconsistencies.. The central question 
for the educational research community concerns the nature of the research 
community's role in educational policy making. % 



Iannacoone, Lawrence (1975) "The Field Study in Educational Policy Research, 11 
- Education and Urban Society , Vol. 7, No. 3, pp. 220-238. 

Discusses the character of the field study and its place in the scientific 
venture, notes the characteristic attitude toward the relationship between data 
and hypotheses held by the field researcher, addresses the requirements of^ edu- 
cational policy research, and explores the fit between these end the field 
study's character. 
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Jenkins, Jerry (1977) Research in Elementary and Secondary Education (Fool' a Gold?) 
Presented at vthe Annual Meeting of the American Educational Research Association 
(New York, JIT, April) 13 p. 

. * v 

This pMper contends that much of what passes as "educational research" has* all 
the appearance, but little of the, substance, of scientific research and that the 
educational coonunlty should realize this fact and condemn the imitation, while 
working to Improve scientifically based and conducted projects. Problems in the 
field center around (1) the mechanisms, attitudes, political structures, and H 
reward systems existing in elementary and secondary education; (2) the time span 
allowed in developing proposals and conducting research and for results to become 
evident; (3) the self-reinforcing attitude of distrust and doubt among educators; 
who are wary of new research because of previous unfavorable research, responsi- 
bility for changing the quality of research rests in frfur basic areaa: (1) the 
financial s^port system, (2) pifrticipatory school systems, (3) the researchers, 
and (4) research training programs. Given the problems involving current systems 
which do not, and cannot, effect rigorous applied research in education and the 
general distrust regarding research as an effective change agent, the author 
makes nine recommendations to Improve the situation, involving (1) organizational 
systems, (2) legal status of research organizations, (3) regional orientation, 
(4) materials and equipment identification and acquisition, (5) model research 
activities, (6) communication and implementation of data, (7) information dis- 
semination, (8) communication and implementation of successful programs, end 
(9) feedback into training programs. 4 

\ 

Keen, Michael (1980) Research and Evaluation in Urban Educational Policy . ERIC/ 
CUE Urban Diversity Series, No. 67 (June), NIE, Washington ? D.6., 25 p. 

A study conducted by the school fjistrict of Philadelphia and that city's 
Federal Reserve Bank, entitled "What Works in Reading',' was used as a case study 
to determine what factors Influence the translation of research findings into 
educational policy. The Philadelphia study, which examinW variables associated 
witW achievement growth in reading, received extensive nationwide media attention 
and was used by the Philadelphia School superintendent's office in the develop- 
ment of recomnendations for changes in reading instruction. Analysis of the 
study's design and of the methods used in disseminating its findings resulted in 
the identification of ten factors associated with its being translated into edu 4 - 
cational policy: 1) identification of the clients affected by the study's find- 
inge, 2) vitality of the Jtopic, 3) participant involvement, 4) technical qualit^T 
5) reporting formats, 6) findings keyed to the decision process, 7) preparatidn 
of policy makers, 8) overcoming resistance, 9) the role of the ombudsman ,^L0) the 
role of the entrepreneur. Factors associated with Research utilization in tirban 
areas were also identified. \ 

\ * 

Ketefian, Shake (1974) Sources of Knowledge Utilization Curriculum Change in 

Nursing Education. Presented at the Annual Meeting of the American Educational 
Research Association (59th, Chicago, Illinois, April) 25 p. 

A fctudy was conducted to suggest an answer to questions in nursing education 
concerning what educational changes are necessary, which directions those changes 
should take, ant* what type of approach should be used to introduce them. The 
study investigated the extent to. which scientifically obtained knowledge is uti- 
lized in the process of change in nursing education, using selected innovations 
as the focus for the study. Information was obtained through a national survey 
of j c H"ent curricular ^novations in nursing as w M ^determined by questionnaire 
and through a small nunber of case studies. The case studies indicated that 
there We various stages in each innovation process for each curriculum change. 
Severa^different modes of identifying the knowledge needed were determined in 
an attempt to classify the approaches used for deriving knowledge. The quality 
of the evidence was not assessed, however. A literature review pertaining to 
sources and methods of obtaining knowledge was utilized. .(A 36-item bibliography 
and a classification scheme for the approaches used in deriving knowledge are 
also included)^ _ 

bo 



KievYt, Mary B. (1975). Vocational Educat ion Research and Development as a Factor 
Influencing Teachers to Change Practices. Washington, D.C.: National Academy 
of Sciences National Research Council, 57 p.. 

With particular attention to vocational research and development, this study" 
reviewed the literature to obtain information on the factors or influences caus- 
ing teachers to change practices. Research and development in vocational educa- 
j!u Wa ! treated as 3X1 independent variable ±n an experimental research design 
witn moderating variables of product characteristics, dissemination efforts, 
teacher education, work context, and teacher characteristics. The sources of 
information included, a number of reviews of research, research monographs, and 
expert opinion statements. Very limited empirically derived evidence was found 
regarding the extent to which vocational research and development has influenced 
' J"^ 618 ,^ change practices. Speculation based on informed opinion concluded 
that. (1) The influence of curriculum development and distribution on teacher 
practice is relatively insignificant; (2) Use of dissemination through institutes, 
workshops, and conferences is inadequate as an influence on teacher practice; 

(3) Teacher education as a mode*of dissemination is of negligible influence; and 

(4) Teacher educators as disseminators of research and development are limited 
in influencing teachers to change practices. Possible explanations for the low 
return in terms of actual change of teacher practice are offered pertaining to 

• organizational factors and to systems for analyzing the characteristics of 
educational innovations. 

I 

* Kleinberger, Sharon (1976) "Research and Policy Making in Education," Studies in 
Educationa l Evaluation . Vol 2, No. 3* pp. 215-26. . ' 

Various educational practices resulting from research recommendations and the 
associated problems are described. The- author suggests that educational research 
can. create a more reliable foundation for policy decisions concerning the intro- 
, Action of reforms, by evaluating their resulting effectiveness. Investigators 
; • . 4 should suggest more pertinent questions and introduce revolutionary innovations. 

• V . " v 

Krathwphl, David R. (1974) "An Analysis of the Perceived Ineffectiveness of Educa- 
tional Research and Some Recommendations," Educational Psycholo gist. Vol 11 
No. 2, pp. 73-86. . ^* — ' 

^ The au^or examines, in the light of funding problems for federal agencies ' 
supporting educational research, the difficulties inherent in the research pro- 
cess as it now exists,' and methods for attacking these difficulties. The thought 
is presented that only by working on these general problems, will the more 
specific problem of funding be effectively solved. Comments are presented on - 
one aspect of educational research: That It is perceived as having little impact 
f upon practice. Explanations of this perception center on (1) The role of 

dissemination, and the relatively little funding' devoted to it; (2) The role of . 
research as a legitimizer of practice, rather, than as an initiator of change; 
(3) The gap between researchers and research users, in which researchers are 
the ^professionals" and users a less knowledgeable "pWic"; (4) The nature of 
scientific models, in opposition to the humanistic concepts employed by users; * 

(5) The problem of "proof*' and "truth" in educational research - when is some- 
thing "proven true" suggested remedies include (1) Increased attention to dis- 
semination; (2) Increased attention to the demonstration of theoretical sources 
of practice; (3) Increased emphasis on research rationale, the development of 
theory, and the integration of research into new wholes; and (4) Attention to 
bridging the gap between researcher ao4 user through increased awareness of the 
gap and its effects, through the use of classrooms for research settings, 
through an emphasis on success in practice as the final prc^pf of research vali- 
dity, and through modification of research development models to include more 
teacher involvement. 
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Leithwood, K.A. (1981) "Managing the Implementation of Curriculum Innovations," 
Knowledge: Creation, Diffusion, Utilization , 2, 3:341-360. 

The thesis of this article is tftat a major problem contributing to the lack of 
systematic knowledge utilization in the school system is the rilglected development 
of demonstrably effective strategies for transforming knowledge into defensible 
courses of action. The author outlines a strategy, or theory of action, consisting 
of a sequence of major^ tasks and procedure's for their accomplishment. 

The framework, with Implementation defined as< a reduction in the gap between 
current and preferred status, consists of three concepts (growth, system, and 
Action) and the relationships among these concepts. For the purpose of developing 
the strategy for managing the relations among these concepts and the implanentation 
process, three tasks are discussed: diagnostic, application, , and evaluation tasks. 

The author describes the above concepts and provides case examples of each task. 



Leithwood, K.A. (1981) "The Dimensions of Curriculum Innovation," Journal *of Curricu- 
lum Studies , 13, 1:25-36. 

The purpose highlighted by this paper is the planning and monitoring of curri- 
culum implementation. The author proposes the concept of an innovation profile 
to aid in this planning and monitoring process. 

One critical task in the development of an innovation profile is the identifi- 
cation and choice of components or dimensions of an innovation. Nine curriculum 
dimensions are identified, the attrjlmfces of each are defined, the nature of their 
contribution to the effects of a curriculum a^re, outlined, and a justification is 
provided for their inclusion in an innovation profile. The paper is intended to 
serve as a brief yet' comprehensive source of dimensions for those attempting an 
Implementation analysis; the criteria to be employed in selecting from the array 
of dimensions those most suitable to a description of a, particular innovation are 
outlined. 

i 

Levin, Henry M. (1978) 'tyhy Isn't Educational Research More Useful?" Prospects , 
Vol 8*f No. 2. <* 

^ The author asserts that the problem of ^usefulness" of educational research 

will be a continuing one no matter what the quality or level of volume of 
research. The author suggests that the problem of usefulness is a function'of 
the different contexts and methods that characterize the research and policy 
x functions. He suggests that the following aspects are important. First re- 
search must address the specific questions of the policy maker. Second, 
research must meet the time requirements of the policy maker. Third, research 
must be written in such a way that it is understandable to people who are not 
experts in educational research. Fourth, the research must take into consider- 
ation the political constraints of the policy maker. The author goes on to 
argue that there should be a tension between educational policy and Research. 
They represent two different cultures with different requirements. 
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Lortie, D.C. (1975) School Teacher , University of Chicago Press. 

The main purpose of the book is to identify the pattern of orientations and 
sentiments which is peculiar to teachers and which distinguishes them from members 
of other occupations. The author argues that the pattern derives from both the 
structure \>f the occupation and the meanings teachers attach to their work. Part 
1 of the book (Ch. 1) is a chronological review of selected structural features 
of teaching (e.g., money rewards, "social position) in which continuity and change 
oyer three centuries of American history is explored. The secondNpart (Ch. 2, 3, 
4) deals with recruitment, socialization, and the distribution of career rewards. 
These processes give .rise to orientations of conservatism, individualism, and 
presentism among teachers. Part 3 (Ch. 5, 6, 7, 8) emphasizes the meanings tea- 
chers give to their tasks and the sentiments they generate while carrying them 
out. The subjects ift&ude teachers' goals, problems of teaching, the general 
sentiments of teachers', and teachers' preferences about their day-to-day interac- 
tions. Part 4 (Ch. 9) centers on three scenarios which might unfold in the future, 

■ f \ " • 
LOU i 8 <' K 'L ( \ 9 , 81 ! ,,Exteraa \ Agents in Knowledg| Utilization," in Improving Schools: 
Using What We Know, (eds.) R. Lehming and M. -Kane. * 

The purpose of this paper is to explore the ways in which external agents 
(e.g., consultants, change agents, linking agents) affect school improvement and 
- knowledge utilization at the local jLevel. In addition, to the synthesis of rele- 
vant literature, the author indicates gaps and deficiencies in the current under- 
standing of the^role of external actors. First, she points out that there is a 
serious lack^f theoretical frameworks in which the study of external roles might 
be located. Second one of the main deficiencies of existing research is the 
emphasis ufSSh a single function of knowledge use (idstrumental decisionistic) and 
a single type of knowledge (externally generated research information) . Third 
the limited context «bf knowledge use research has also placed limitations on the 
research about the role of external agents. Research investigating the role of 
external agents has tended to view the external agent largely within the context 
of the technological push" framework. The author argues that research on strate- 
gies and tactics must be firmly based in a framework for understanding knowledge 
use settings at the local level and in an improved understanding of "demand-pull" 
based knowledge utilization as well as "technological push." The author recom- 
mends that more attention be paid to the ways in which organizations act as 
external agents (e.g., modeling organizational information networks at the user 
level). In addition particular attentidn should be paid to organizations which 
are conscious agents, those that have power relationships with school districts 
and those that have a potential for becoming more consciously client and informa- 
tion-use oriented at the institutional level. 

Madey, D.t. (1981) "Some Benefits of Integrating Qualitative and Quantitative Methods 
in Program Evaluation, with Illustrations," Educational Evaluation an d Policy 
Analysis (in press) . 

This article advocates the integration of qualitative and quantitative methods, 
illustrates the argument through a discussion of a federal educational program 
evaluation, the State Capacity Building Program, and summarizes the benefits and 
costs of such methodological triangulation. 

The article's major thesis is that ,the integration of quantitative and quali- 
tative methods within a single evaluation ha's synergistic effects in the three 
major phases of the research process; design, data collection, and analysis. 
After reviewing the historical debate between advocates of the two methods, the 
State Capacity Building Program and its evaluation are briefly described. The 
article deals first with the contributions of qualitative methods to quantitative 
work; 'then, conversely, the contributions of quantitative to qualitative methods . 
are examined. A separate section explores some of the costs involved in such a 
methodological marriage. In a concluding section, the applicability of integrated 
methodologies to other evaluations is explored. 
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Mahan, J.M. (1972) "Frank Observations on Innovation in Elementary Schools," Inter- 
change , 3, 2-3:1^4-160. — — 



Instructional innovation in the elementary school often is reported in educa- 
tional literature as superficial and/or compromised. This paper presents 
curriculum installation guidelines and recommendations for consideration by 
educators responsible for effecting intended curriculum change. Based on inten- 
sive field experiences in more than 60 schools of diverse characteristics, and 
supported by survey responses of innovating administrators and teachers, the 
guidelines reveal actual implementation problems that impede and endanger curri- 
culum change efforts. Educators involved in curriculum change are encouraged to 
transform their theoretical models for change into concrete, replicable activities 
(work) and events that directly deal with the field problems that stimulated the 
guidelines. 'References are made to a series of curriculum installation projects 
that were progressively more successful because the guidelines were formulated 
and made operational in the field. 

(author modified) 



fcC A£&it«tSS i Jf a uS "H™ ?• Gillespie (1973) Information Needs of Local 
Administrators of Vo cational Education.. Final Report — uhio State Universi ty, 
Columbus, OH. Center for Vocational and Technical Education. National 
Institute of Education (DHEW) , Washington, D.C. 

Research findings often fail to reach key decision-makers either in the 
proper form or in time to assist in decision-making. Also, little is known 
about major problems for which local directors need information, methods of 
seeking information used by these decision-makers, information normally used, 
or the criteria employed in selecting information. The purpose of this study 
was to identify the critical problems of local administrators of vocational 
education and the information sources they utilize. Data were collected from 
certain local administrators, selected by a stratified random sampling tech- 
nique, through the use of telephone interviews and a mail questionnaire. Major 
conclusions include; (1) Local administrators generally perceived little need 
for information for use in problem resolution, (2) Most decision-making was in 
the absence of an information search, (3) Respondents desired experienced 
people as their major information source, akd (4) The major criteria for utili- 
zation af print materials were the familiarity or degree of experience of the 
administrator with the materials, and the content quality. 

Menges, Constantine Christopher (1978) Knowledge and Action; The Use of Social 
Science Evaluations in Decisions on Equal Educational Opportunity, 1970-73 , 
National Institute of Education (DHEW), Washington, D.C, 73 p. 

This report analyzes why the federal government has made little use of 
evaluation studies and data available on equal education and school integration. 
An overview is provided of federal policy and action on equal education from 
1970 to 1972. The utilization of evaluations in the areas of educational and 
social effects of desegregation, criteria/incentives for maximum feasible deseg- 
regation, and cotopensatory education and achievement are discussed. Also traced 
are political, bureaucratic, and intellectual factors which affected the extent 
to which evaluations shaped federal policy. It is held that with more communica- 
tion between legislative staff and researchers, valid social program evaluations 
can be chosen- and cumulatively synthesized, in order to apply them to decision 
making and policy. 
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Miles, M.B. (1981) "Mapping the Common Properties of Schools," in Improving Schools : 
What We Know. <ed.) R. Lefiming. Santa Monica, CA: Sage. 

^ The purpose of this paper is to summarize what seems to be known about the 

common properties" of schools on an empirical basis, to assess the explanations 
offered for the presence of such properties, and to outline an agenda for future 
research, both to provide more descriptive data and to test competing explanations 

The common properties of schools, as abstracted from a thorough literature 
review, are discussed according to nine dilemmas encountered by school systems. 
These dilemmas are created by the interaction of schools* common properties and 
the primary tasks of schools, and occur at all levels of the scho9l system. Each 
dilemma is explained, a critical review of regularities is presented, causal ques- 
tions are addressed, and a listing of knowledge .gaps (issues where more empirical 
data are needed) are discussed. 

Miles, M.B. (1965) "Planned Change and Organizational Health: Figure and Ground," 
in Change Processes in the Public Schools . R.O. Carlson, et al .. Eugene, OR: 
Center for the Advanced Study of Educational Administration, University of 
Oregon: 11-34. 

The author contends that those concerned with innovation have tended to focus' 
on the effectiveness of the innovation itlelf and to neglect the readiness of a 
specific organization to absorb the innovation. The author attempts to correct 
that deficit and identify factors of organizational health in relation to utili- 
zation of innovation. He formulates nine dimensions of organisational health. 
The author notes, however, that educational organizations have certain properties 
which tend to diminish their organizational effectiveness. These include goal 
ambiguity, input variability (i. e. , they must accept a wide variety of students), 
role performance, invisibility, low interdependence, vulnerability, lay profes- 
sional control problems and low technological investment. The author then sug- 
gests specific interventions to improve organizational health: (l^team trailing* 
(2) survey feedback; (3) role workshop; (4) target setting and supporting activi- 
ties; and (5) organizational diagnosis and problem solving. These five means of 
intervention have the following common principles: self study, relational empha- 
sis, increased data flow and expert facilitation. 

Miles, Mathew B. (1964) "Innovation in Education: Some Generalizations. " In M.B. 
Miles (ed.) Innovation in Education , New York; Bureau of Publications, Teachers ' 
College, Columbia University. 

This paper discusses innovations in education. Although the merits 6f , the 
innovation itself are rarely the major determinants with respect to adoption, 
some properties of the innovation do have a bearing on adoption. The following 
are some properties which affect adoption: 1) innovations requiring inordinate 
outlays of time or money by the adopting group are likely to move slowly; 2) ease 
of discard if the innovation is unsatisfactory; 3) congruence with the potential 
adopting system. Most innovations appear to be stimulated by some active person 
or grou^. Innovators tend to be in positions of. authority and have contacts 
with co-workers. Gtoup sponsored innovations tend to be moderate and implemen- 
tat ion is often facilitated by creation of a temporary system. Relevant char- 
acteristics of target organizations include the strength of the forces working 
for stability, and cultural attitudes affecting change. The process of imple- 
menting change needs careful planning and should include attention to diffusion, 
reduction of pressures on decision makers and use of linkage. Two models of 
change are discussed: 1) one formulation begins with development of interest in 
an innovation and ends with a decision tp adopt, adapt or reject; 2) another 
formulation begins with criticism of existing programs and ends with action on 
decisions about the innovation. The optimal conditions for change to take 
place are: design requires alprotected, autonomous environment; diffusion is 
best done dispassionately and credibly; visits to observe operation of the 
innovation and potential adopters to evaluate the innovation; during adoption 
the user needs support. 
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Mitchell, Douglas E. (1978)- Social Science Utilization in State Legislative Policy 
Making. Presented at the Annual Meeting of the American Educational Research 
Association (Toronto, Ontario, March 27-31) 25 p. \ * 

In this paper, social science utilization* is studied as an aspect of legisla- 
tive decision making. The purposes of the paper are to provide a review of 
political science literature on state legislatures, offer a theoretical frame- 
work for interpreting utilization, of social science research, apply the frame- 
work to interview data, and generate a theoretical framework to test additional 
social science utilization hypotheses. In 1977; nearly iOO legislators, staff 
members, executive branch personnel, and interest group spokesmen dealing with 
educational legislation in Arizona, California, and OregoiKwere interviewed 
regarding legislative issues and influences. Data from th&e interviews 
suggested that policy makers continually confront situations in which pressures 
to act are diverse, incessant, intense, and difficult to identify clearly. A 
theory of social science utilization was developed which was in accordance with 
data from these interviews and from literature on state legislatures. The 
theory was based on three major elements; 1) Location of points within the 
legislative policy-making process at which social science findings might be 
expected to have a significant impact; 2) The linkage system which ties social 
science research to utilization in the policy process; and 3) Orientational 
frames of reference which influence legislators to utilize social science and/ 
or other resources in decision making. Hypotheses based on this theoretical 
framework will be used to test the nature of legislative issues and the back- 
ground and attitudes of legislators. 

l * 
Mitchell, Douglas E. (1980) "Social' Science Impact on Legislative Decision 

Making: Process and Substance," Educational Reaearcfrer . November 9-19. ^ % 

In this study, data were collected from 160 key legislative policy influ- 
entials in three states. Each respondent was asked to: 1) describe recent * 
legislation in one or more of the six issue areas; 2) identify the key actors 
who formulate and influence these legislative decisions; 3) discuss how the 
legislature is influenced by these key actors; 4) define "the issues" being 
resolved as legislation is enacted; and 5) identify the resources utilized to 
control or influence the decision making process. Based on their interview 
data, the author defined four distinct stages of legislative policy making: 
articulation; aggregation; allocation; and oversight. The author suggests 
that social science serves different intellectual and Social functions during 
each of the four stages: During the articulation phase social research find- 
ings make intellectual contributions through policy problem definition and 
fulfill the social function of building group solidarity within major inter- 
est groups. During the aggregation phase research makes intellectual contri- 
butions through persuasion and coalition building which the author refers to 
as problem solving. The social function of research findings at this phase is 
the creation of a coalition of support. During the allocation phase the 
intellectual function of research is achieved through evidence assessment. 
Socially, utilization at this stage means debate winning. During the legisla- 
tive oversight phase the major intellectual contribution of social science 
research is one of performance evaluation. The social science utilization, in~ 
this phase, tends to become a vehicle for the mobilization of criticism. On 
the basis of his data, the author makes three basic conclusions regarding 
social science utilization within state legislatures: 1) social science is 
only one of four distinct types of expert authority influencing legislative 
decisions; 2) the utilization of social science contributes to both intellec- 
tual and social aspects of decision making; 3) the contributions of social 
science shift as legislative proposals move through the phases of decision 
making • 
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Nash, Nicholas and Jack Culbertson (eds.) (1977) Linking Processes , In Educational 
Improvement: Concepts and Applications . Goltjnbus,- OH: University Council for 
Educational Administration. 



This volume is designed to achieve several directives - to produce for edu- 
cational leaders an up-to-date synthesis on the role of linking agents and 
agencies in educational improvement activities and, in the process, to identify 
and discuss important knowledge utilization issues of interest to the research, 
development, and training communities; to address the immediate realities that 
internal and external linkers cotffront and to shed light on the kinds of organi- 
aational, human, and knowledge resource^ available to them; and to prqyide better 
bases for advancing linking through new fclans ,and developments. The organization 
of this volume is intended to illuminat^ the role of those linking agents engaged 
in or providing support for improvement in }ocal educational agencies, either 
internally or externally. The volume Synthesizes key concepts and findings about 
change processes, uses of knowledge in change, the management of change, the 
functions of linking agencies in Improvement activities, the attributes and skill 
clusters needed,by external linking agents to function effectively, and a national 
training system for both internal and external linkers. 

Orfield, Gary (1976) "Research and the Antibusing Debate." Law and Contemporary 
Problems , V 42, No. 4, pp. 141-73. 

Examines several obstacles to the effective use of social science research in 
school desegregation controversies and discusses the nfeeds of judges, administra- 
tors, and elected officials whomust make decisions concerning desegregation ^land. 

Paul, Douglas A. (1977) "Change Processes at the Elementary, Secondary and Post- 
Secondary Levels of Education.". In Nash, N. and J. Culbertson (edaf.) Linking 
Processes in Educational Improvement: Concepts and Applications. Columbus, OH: 
University Council for Educational Administration. 

The purpose of this review ijs twofold - to summarize the major models of 
educational change and to present a series of generalizations based on recent 
empirical studies about educational change. In the first section, previous 
reviews of the change literature are presented in order to establish a broad / 
perspective concerning research iirlhis ^rea. Three dimensions- of change- 
processes, influences, and effects-derived from previous research and from the 
empirical studies cited in section 2 are proposed as a succinct means for classi- 
fying the wide variety of research on change. Four models of change-problem- 
solving, social interaction, research-developmetffc^dif ftfsion, and linkage are 
described and compared using the three dimension*. J Studies associated with the , 
models are cited and a summaiy Of the models is^M^empted by using conceptuali- 
zations about change strategies and Images of practitioners. In section 2, over 
100 empirical studies completed since 1970 are analyzed in order to provide the 
basis for a series of generalizations and inferences about change.. Each genera- 
lization or inference is Judged as "firm," "moderately speculative," or "specula- 
tive." These judgments accompany^aach generalization. A bibliography is included. 
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^.l^nfr i 19 l 5 \ A Conceptual Framework for S tudying "Knowledge Utilization" . 
SSS! 5 q. ^ril""!" 8 ° £ ^ AmerlCa ° EdUCiti0nal Re "" ch Aasoclatloa 

This paper, written from an organizational perspective, begins with an 
examination of the deficiencies of knowledge utilization and educational change 
literature. It suggests the explication of descriptive and heuristic conceptual 
dimensions as one approach for facilitating a descriptive perspective of know- 
ledge utilization. The efficacy of three descriptive. and heuristic dimensions 
(systems, processes, and characteristics) are explored, along with a fourth 
dimension, utilization, which Is the dependent variable in many studies of know- 
ledge utilization. A framework for classifying knowledge utilization research, 
based on these four dimensions is constructed. The paper contains a bibliography 
of educational change literature. «""8«pny 

Paulston, Holland G (1979) Paradigm Conf lict in the Cessment of Education ,! 
Reform; Evaluation as Social Critique . Wf. 

• « 

Traditional quantitative-experimental models of evaluation have failed to 
contribute to ■ educational re-formj they should be supplemented by qualitative- 
subjective models, which can be grouped under the rubric of naturalistic, or 
non-experimental evaluation. This naturalistic approach does not attempt to 
be yalue-free; rather, it recognizes that evaluation is highly political, 
trying to balance the conflicting values of evaluators, policy-makers, and 
. educational planners, by stressing context rather than method. Existing evalu- 
ation models can be grouped under two research paradigms, which relate^ocial 
Sff " t S?° r i M - C ° educatio ^ 1 reform: Equilibrium/liberal and criticsl/con- 
J , " 18 a8Bocl * ted traditional criteria, like measurement, 

cost-effectiveness, and social efficiency; the latter assumes political con- 
flict and stresses balance of power and social justice. Conflicting theories 
of social change influence both the definition of educational reform aril the 
choice of evaluation method, standard, and outcomes - a distinction which M 
metaevaluation has, ignored. In conclusion, a dialectic strategy, is recommended 
wi?Jir£radig^s Can Ch ° 08e b6tWeen tne two Pa«diff»« and among the methods 

Pellegrin, Roland J. (1965) "The Place of Research in Planned Change." In R.O. 
Carlson, et^al. Change Proc esses in the Public Schools . Eugene, Oregon; 
Center for the. Advanced Study of Educational Administration, University of 
Oregon, pp. 65-75. ' 

In this article the author discusses the need for reliable research as a * 
basis for planned change in education. According to the author, educational 
decision makers lack appreciation and understanding of the nature and value of 
research and rely heavily on authority to justify existing practice and policies, 
intuition and common sense seem to be the major premises for innovation. The 
author suggests several obstacles which must be overcome before educational 
research can provide reliable knowledge on which to, base policy, practice and 
innovation These obstacles are; a widespread lack of appreciation for and 
understanding of the nature and value of research; much existing research is 
low in quality,- weak in the insight it Imparts, and of dubious utility to the 
practitioner; the nature of functions of "theory" are poorly understood. For 
many practitioners, the term is an antonym for "practicality"; there, is consi- 
derable confusion about the relationship between empirical fact and values; 
research on topics important to- education covers a wide range, many of which 
are complex and difficult to investigate. The author asserts that for educa- 
tional research to be reliable enough to be used as a basis for practice, it 
must be either: 1) the testing of well-defined but isolated hypotheses or, 
I) research directed by systematic and integrated theory. 




Pincus, John (ed.) (1980) Educational Evaluation in the Public Poli ty Setting 
Santa Monica, CA, Rand Corporation. 

This monograph is composed of 5iessays which criticize from federal, state 
and local perspectives, current methods of evaluating government sponsored 

OnZVSJSF'?' A recurrin « theoe i8 " that experimental Resign methods 

do not provide adequate information for policy makers' needs. .Chapter One of - 
the report summarizes the main findings of Chapter Two through Five, and dis- 
cusses a possible new approach -to evaluation through the social effects of 
• government programs. Chapter Two is a. critique of the experimental design 
approach, with recommendations for new perspectives on evaluation. Chapter 
Three analyzes federal program evaluation from the local perspective, and argues 
that present evaluation methods are generally not useful from the local per- 
spective. Chapter four describes an Executive Branch attempt to evaluate Title 
I of the Elementary and Secondary Education Act on behalf of Congress. The 
strategy adopted was to define the aims of • devaluation in light of Congress's 
policy making authority and concerns. Chapter Five discusses how education pro- 
grams can be more successfully evaluated at the state level. These essays offer 
a generally consistent set of views on the current state of program evaluation 
m the federal education system. 



Pincu»,.J. (1974) "Incentives for Innovation in the Public Schools," Review of 
Educational Research . 44, 1:113-144. ? 

i 

This paper sets out conceptual propositions about the structure and incentive 
systems of public schools as they relate (1) to the adoption of innovations and 
. U) to their Implementation in the schools. These propositions have systematic 
implications for educational R&D policy as well as for such broader questions as 
how to implement planned change In bureaucracies. 

The general thesj.s of the paper is that the, market structure of the public 
school as an industry" has a major effect on schools' decisions to adopt inno- 
vations, and the bureaucratic structure and incentives of schools shape in 
specific ways the transition from adopting innovations to implementing them. 
The objective of the paper is to lay the groundwork* for a testable theory which 
integrates the incentive effects of both* market structure and bureaucratic struc- 
ture. / 

'The article concludes that given competing demands in the public utility market 
of the schools, diversity of approaches to schooling will have to take place pri- 
marily in a public marketplace, and that this diversity, rather than the preserva- 
tion of the existing institutional order, should be the focus of R&D policy. 

This paper moves in the direction of specifying a model of how school systems 
behave in response to opportunities' for change. The author suggests five broad 
emphases for R&D policy in encouraging adoption of educational innovations: 

1) large-scale experimentation; 

2) collaboration between R&D agencies and educational leadership networks- 

3) case studies of successful and unsuccessful innovation; 

4) research that will improve the R&D* community ' s understanding of the existing 
pattern of incentives in the public schools; .existing 
trying out methods of restructuring system incentives (e.g., changes in mar- 
ket structure; changes in locus of control; changes in individual incentives; 
and clear standards of accountability better information systems). 
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Pi °Ed«2S» » 19 llL2? Ch °l J dapt f! tlon ° f Re " arch Flnd W trough Inservice 
Education. Washington, D.C.: Teacher Corps, Office of Education. •. 

A program designed. to adapt educational research findings to practical 
lltlZl™ US J i :/ e8crlbed -- A ™^*y of option, for adapfing rL«c5 through 
^hn^T " ? t! a J i0 ? *" de8crlbed ' includ« formal university courses after 

llndlnt IluT e J- P«« r — • °" '° one and small group consultations, inae* 
?e^m pJannJni « 8rad r the8M ' vl8lt * tio **. Professional conferences, ' 

ttZ It'™ 1 ** ^l****' fnd support for teacher action research and pilot pro- 
gram proposals Proposals for action research and , Hot programs are supped 
by the prolect for materials, consultative services, and publications and " 

pac: a em o? a t t e i achers The ^ t0 accon »° d ^ d «<e«nt styles and 

Powell, Marjorie (1978) Utilization of Research by « PnH 7 - M .w. ff Regulatory 

SSZV* A l^ ' * ,resented at Annual Meeting If th ^ Ameri can Ed uc a- 

tional Research Association (62nd, Toronto, Ontario, Canada, March 27-31) 18 p. 

fnr T 5 e California Commission for Teacher Preparation and Licensing, responsible * 

condicMir." r ; ifiCati0n ?* aPPrOVal ° f ° f ed^cat^ £ 

conducting a major research effort to identify teaching behaviors tha^are 

tltlnnS^V"™ 1 ** ° f " adin8 "-^hematics in the element grades- 
SicE tea ?J" e r* lu * tlo ^"« d y. while the results will not indS.fe 
rea"Sv Sv! JiS P ? • ltern * tlvM » h ° ald ^ adopted, they will have, anS .1- 
noMo L S" l0PaCt ° n the thou « h " «d actions of the member, of the 

policy "king body Policy maker, have modified their conceptions of teacnin. 
and teacher education; learned about the re«earch proce.. and u.ed re.e«ch to 
an.wer other que.tion.; used result, in their individual worked nea" "Se 
results quoted by individual, arguing, for specific policy decisSn. "n addi- 

of lL °" t° u the r «8 ul * tlo °- of the Commi.. ion have become more aware 

of and concerned with the results of research on teaching. Specific «ampHi 
are discussed to elaborate each of these uses of the research results 

SCh ^s^* G ' ^SV'l 1 (ed80 U972) The Oregon Studies in Educational 

Research, Development. Diffusion, a n d Evaluation^ Volume III; Ca nr.nr„ .i 
works for Viewing Educational RDD&E. Fina J^ggport . Was hington, B.C.; National 
Center for Educational Research and Development (DHEW/OE) 

This volume represents the output of a yearlong effort to clarify and firm the 
conceptual base that underlies educational RDD&E . It contains three commissioned 

" authored ^ Drs. Henkrik D. Gideonse, Gene V. Glass and Blaine R. Worthen, 
and by Leslie J. Briggs and one paper prepared by H. Del Schalock and G. Roger 
Sell of the staff of the Oregon studies. The volume also contains a formal 
critique of each paper, an author response to each critique, and an introductory 
and summary critique of the papers as a set. Individually, the papers deal with 
°?!u° r u m0 ! e a8pect * of the domaln of educational RDD&E. Collectively, they deal 
• with the domain as a whole. As a set, the conceptual papers and their critiques 
are intended to serve as (1) a benchmark document with respect to the conceptual 
structures underlying educational RDD&E, (2) a primary reference for persons 
entering the field of educational RDD&E, and (3) a stimulus to the continued 
dialogue that these papers have only begun. 
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Schmuck, R., Runkel, P., and D. Langmeyer. (1969) "Improving Organizational Problem 
Solving in a School Faculty," Journal of Applied Behavioral Science . 5:455-482. 

* • 

The purpose of this project is to test whether Improved organizational problem 
solving could be produced in a school faculty by training in interpersonal commu- 
nication skills. The intervention is pointed toward organizational development 
and remained fixed on organizational roles and norms and their interrelationships. 
The major training goals are developed out of a conception of flexible organiza- 
tional problem solving (communication, participation, etc.). Evaluation of the 
effects of training show a need for applying techniques learned and also a quality 
of relationships improved. In final evaluation the strengths and weaknesses of 
the intervention are discussed. In conclusion, the authors contend that this 
project is salutary for a school faculty and contains valuable lessons for consul- 
tants or change agents. 

* 

Schmuck, R. (1968) "Social Psychological Factors in Knowledge Utilization." In 
T.L. Eidell and J.M. Kitchell (eds.) Knowledge Production and Utilization in 
Educational Administration. Eugene, Oregon: Center for the Advanced Study of 
Educational Administration, University of Oregon. 

The focus of this article is a theoretical analysis of problems of knowledge 
utilization in education. Based on this analysis, the author develops a plan 
for training administrators and staff to be more receptive to research. Accord- 
ing, to the author, social relationships between behavioral science researchers 
and administrators are characterized o y hostility and resistance which impedes 
Utilization. The author suggests that there are ten social psychological assump- 
tions which must be considered in planning action to facilitate research 
utilization. Included among these are: there must be trust, openness and 
.attraction between the administrator and the researcher if there is to be effec- 
tive communication; cooperation activity should be structured so that each will 
benefit directly from it; at the start of any collaboration, there should be a 
discussion of the forces which might Impede either side's participation. The 
author then outlines ten stages which should be present in the design of a 
training program for the development of an individual administrator. In addi- 
tion, the author suggests eight stages for modifying the faculty's expectations 
and pressures that support the status quo. 



Short, Edmund" C. (1973) "Knowledge Production and Utilization in Curriculum: A 

Special Case of the General Phenomenon ." Review of Educational Research, 43. 
. 237-301. : 

This paper surveys the general literature on knowledge production and utiliza- 
tion and explains how this field differs from more pragmatic approaches. It 
deals with three domains of interest: knowledge production, knowledge utilization 
and the relation between the two. The autWor examines each domain of Interest 
from three perspectives: 1) studies related to KP&U in general, studies related 
' to education in general, and work more specifically related to curriculum. He 
provides an extensive bibliography encompassing all three perspectives. , 
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Sh °n?A ^?T d C ' (1973) " Tasks of Technical and Public Policy Experts in Educational 
R&D • 2 7 p . 

In order to develop and implement new governmental policies in educational 
research and development (R&D) which meet the needs of R&D specialists and the 
concerns of^ the public, it is important to understand the complexity of the 
research-development-knowledge utilization cycle and to' develop a strategy which 
serves the public interest but reserves to the R&D specialists those decision* 
and technical activities which only they can adequately carry out. The guide- ' 
;rn e V f ,! UCh * 8trate «y are that a) A philosophy of the government's role in 
R&D should be developed and promulgated through appropriate legislation; b) A 
permanent secretariat, responsible for developing and implementing the role 
statement formulated, should be established by legislation; c) the educational 
R&D community should be granted financial assistance at the local, state, regional, 
and national levels to establish its initial organization and governing procedures, 
for the determination of its priorities, for the identification of continuing 
sources of revenue for the support of research, and for the dissemination of the 
results of R&D work; d) Each R&D institution should receive unencumbered grants 
of money for allocation to specific knowledge-producing project applicants; e) A 
system by which knowledge can be readily communicated to the practitioner should 
be developed; and f) No policies should be set which diminish the effort that has 
to be given to studies related to the ongoing operation of particular systems or 
programs, 

N • 

Sieber, S.D. (1981) "Knowledge Use in Public Education: Incentives and Disincentives " 
in Improving Schools: Using What We Know , (ed.) R. Lehming and l4. Kane. Beverly ' 
Hills: Sage Publications. 

In an attempt to study knowledge use in education, this paper takes into 
account the perspectives, sensibilities and aspirations of school personnel in 
various settings. These considerations provide a paradigm for studying educational 
change which requires understanding incentives for change, in general, and incen- 
tive systems in educational settings, in! particular. This paradigm assesses 
incentives and disincentives at both the individual and the organizational level. 
Among various classifications of incentives and disincentives, this paper mainly 
employs a taxonomy characterized by external vs ., internal sources of knowledge 
use incentives/disincentives in educational change. In general, this paradigm 
attempts to understand the customary rewards and changes that threaten these 
rewards (knowledge use disincentives)', on the one hand, and those ways in which 
knowledge use efforts might contribute to acquisition of preferred benefits (know- 
ledge use incentives). Finally, the paper briefly refers to some major dimensions 
for depicting the incentive structure of local settings. 



Sieber, S.D. (1974) "Trends in Diffusion Research: Knowledge Utilization," View- 
.Efeints, Bulletin of the School of Education, Indiana University, 50, 3:61^8lT~ 

The author discusses and critiques the major models and conceptual schemes for 
achieving knowledge utilization in education. Hfe notes a number of "points of 
confusion in the existing schematizations which tends to give an impression of 
competition among models of planned change. The author points out that very 
little is known about knowledge utilization from the viewpoint of the user. For 
this reason, he advocates a user oriented approach to knowledge utilization. 
According to the author, ah example of such an approach is the Pilot State Dissem- 
ination Program. Based on the author's evaluation of this program he contends 
that enlightened educatipnal reform at the local level depends upon three factors. 
The first is access to the national pool of research and other relevant informa- 
tion. Second is extension agents who are able' to help clients determine their 
needs over time and to help them in using information addressed to these emerging 
and changing needs. Third is the opportunity for clients to define their own 
needs and to select from a variety of alternatives. 

[C . * 



Sieber, Sam (1968) "Organizational Influences on Innovation Roles" In T.L. Eidell 
and J.M. Kitchell (eds.) Knowledge Production and Utilization in Educational r 
Administration . Eugene, Oregon: Center for the Advanced Study of Educational * 
Administration, University of Oregon. 

The author identifies four features of the public education system which 
distinguish education from medical practice, industry and agriculture, and thus 
effect the applicability of diffusion research^n^ these fields for ^ucation. 
The four distinguishing features of educational structures are: vulnerability to 
the social environment (adoption of innovation depends more upon political feasi- 
bility than educational value;) che professional self-image and associated values 
of educational personnel (the insecure professional self-image of teachers * 
increases resistance to change and mitigates against^ fconmunication of innovation); 
the diffuseness of educational goals (terminal goals okeducation and measurement 
of their attainment are difficult to specify); and the nfcod for coordination and 
control of the primary clientele as well as of the employeesW the system. The 
implications of these features for diffusion and Innovation are discussed and 
suggestions are made for strategies of change. Three classical strategies f pr 9 
inducing educational change are identified and analyzed: ... the Rational Man Strat- 
egy (one-way conmunication is adequate); the Cooperator Strategy (involves parti- 
cipation of members of the system) ^and the Powerless Participant Strategy 
(assumes that practitioners are unable to make major changes in the educational 
structure). The author suggests that wJjat is needed is a status occupant 
strategy of, change which takes into account the distinctive characteristics of 
educational structures, and the conditions under which practitioners will respond 
to the tactics applied in each of the three traditional Strategies. In addition, 
the authot suggests that reorganization appears to he necessary at both the local 
and national levels. \ 

, • • ;j 

Spooner-Smith, Laura, et.al . (1975), User Needs and the Development Process . Pre- 
sented at the American Educational Research Association Annual Meeting (60th, 
Washington, D.C., March 31-April 4) 21 p. 

An educational development team, working at the Center for the Study of 
Evaluation to produce the Elementary School Formative Evaluation Kit, saw that 
its mission was not simply to show that a product meets pre-determined performance 
specifications but was also to ensure that a product can be readily adapted by 
/ the user td the "noisy" conditions under whioiTit ultimately will be V used. 

People are- likely to accept an innovation if it is consistent with their profes- 
sional nortoa, levels of *aq>ertise, and day-to-day responsibilities. It is, there- 



fore, the developer's fun£« 
that takes into account the 



:on to demise a technology for solving a given problem 
"intentions and behaviors" of the target audience. 



Spiick, D.W. (1974) "Reward Structures in the Public High School," Educati onal Admin- 
istration Quarterly . 10, 1:18-34. 37 

This article provides a literature reivew on taxonomies of incentives which 
induce cooperative behaviors from organizational members, with the concept of 
incentives or rewards being viewed as multidimensional in nature. The author 
then studies the relationship between eight categories of rewards available to 
teachers in high schools, and teacher behaviors of absenteeism, recruitment, and 
retention. . The findings emphasize the importance of intrinsic motivator* (those 
rewards the individual derives from the work itself) and suggest basic differences 
in motivational patterns between professional and production oriented organizations 

£ ■ ' Hi 



£™Sw « ( i ? } ^ g Reaearchars WQ'fciM with Linking tent* ^ Bring 
lilZ, » SC k°^ 19 - at the Annual Meeting of theAmerican Educa- 

tional Research Aaaociation (62nd, Toronto, Ontario, Canada, March 27-31*23 p. 

for^'..^" 8 ^ 6 Project ° n Di"«nination specified four components as critical 
iZntlvl a lnf0 ™* ti0n dl "-*»"o«i system: linkage, information, 

incentive, and management. An analysis of Pennsylvania' system, using this pro- 

functJonJnl* rr" 6 ? " Veral Pr ° bleMS X) The.link.ge component was not 

f«J S fSX\ 2> a education agencies could not adequately identify needs; 
• Sf P } The " ,w «% no -Sents in place, , In response to this situation 

i*!^'^ 7 h ^ Ql Ifflproveaen,: Program (PSIP) was developed, linking the 

' ™! ^Pjrtment of Education, two intermediate units and tnree research 

sSooS! o ?± i°r ! Ch °° lfl - 1118 "■•*«* agencies are Research for Better 

l-rll i ^ Infornation Services for Education/ and the Learning Re^ 

search and Development Center of the University of Pittsburgh. PSIP is built 

^f 6 ™^ 1 *^ linking agent who facilitates two-way communication » 

su^^J J OC *i « f "J reflearch outcomes and products. This linker is 
SETS. F ^ ! ! fr S™ he " 8earch •« encie » *«d by a school-selected team. 
There are four major PS IP strategies: need, definition, knowledge consoUda- 

;„°?: a "" a " h outcome "Action, and implementation. First year evaluations 

^irlfr^ 8 ° n " 8lte inv °l v «» ent of the linking agent is essential to 

maintain the momentum of the PSIP process. J X 

'^Jeco^^M R ' r d „°^ en ^ ^° bert G ' (1968) Mlm P* ct of R *" arch Flings and 
Recommendations in Urban School Districts: A Case Analysis." Presented at the 

American Educational Research Association Meeting (Chicago, (Illinois, February), 

~ T„ th 8aP K eXifltfl i et T! n educat ' iona l research results and new educational programs, 
to J5/£?r: l ™ Creese ^ the supply of professional change agents 

rLf^K 8 P w° T * f 0 !""*" 00 of applied development ceaters, it is the 
researcher, or the professor, who must, along with teachers and administrators. 

shiws thirrjf^' 0 J 1181 ":" 9olutiona C ° Auction.! probfems. This case study 
shows that individual professors can be effective in helping to bridge the gap 
between research and practice. »r«»Be cne gap 

'^^rSewLds'-'Edu^M 0 ^? 1 !! IT* 108 Pr " tic " : Their Ration to System Norms 
and Rewards, Education al Administration Quarterly . 10, 1:35-43. 

nri^V^l J* « ""ndary analysis of data collected' in 1966 during the Cooper- 
fiJlrnH Je ? ? ? ducatloM l Development (COPED), a research and development 
enterprise involving several school systems and five universities: w 

Teapher behavior, in this case ths use of innovative classroom practices was 
examined with emphasis on its relation to norms ( informal properties of asocls! 
system) and to rewards (formal properties of a socTaTlyTtL) for innovativT 
teaching. 

fl vJ^J e8U \ t8 I"*" -00 " 4 the ^Portance of viewing individual behavior in its' 
systemic context: innovations occurred where teachers saw them/ as rewarded 
independent of individual dispositions toward such behavior, lis su^es^ 'thst 
c^n^ Wl11 te " *-«« -1— « *» -ny^X 

Tyler, Ralph <1980) "Why Research is Needed in Education," UCLA Educator . Vol 21, 



Calls for increased and systematic efforts in educational research to provide 
for meeting the new tasks of educstion. Enumerstes five types of changing Gon- 
dii Si ! Ch °° 18 aDd COlle8M tod * V - Emphasize'that educational 
f^f!?",' *1 "search, and wide utilization of new knowledge are 
essential to the attainment of new educational objectives. 
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Tyler., Ralph W. (1978) "How Schools Utilise Educational Research and Development" 
In R. Glaser (ed.) Research a nd Development and School Change . Hillsdale, NJ: 
Erlbaum Associates. ' : 

' ' •■* 

Teachers and administrators do not generally report that they use the results 
of educational research in their .work, One explanation for this is the cumula- 
tive nature of systematic knowledge, which means that much of what is used today 
was formulated in the past (e.g., Pia^t and Thorn*ike) . When new knowledge is 
Incorporated into the practice of a teacher, he or she rarely is aware of the * 
source and thus it is unlikely that the practitioner will appreciate the contri- 
bution that research has made to their work. The author suggests that two very 
. common uses of research are: 1) in forming the conceptions that make up their 
cognitive maps and 2) in the generalizations that guide the conscious perfor- . . 
mance of their professional tasks. According to the author, it is unlikely that 
new knowledge will be incorporated into the practitioner's map unites they meet 
one or bbth criteria - "consistency"with the reality they perceive Vd/or pro- 
- . viding guidance for more .'effective performance of their tasks. The Author 

suggests that more attention should be given to the problems that practitioners 
face in trying something new. ' . ; 

Weick, K. (1976) "Educational Organizations as Loosely Coupled Systems , " Admin istra- 
tive Science Quarterly , 21:1-19. 

In contrast to the prevailing image that elements in organizations are coupled 
through dense, tight linkages, it is proposed that elements are often tied to- 
gether loosely. Parts of some organizations are heavily rationalized but many 

. ■ parts also prove intractable to analysis through rational assumptions. ,lt J.s 
this substantial unexplained reminder that is the focus of this paper. Loose 
coupling lowers the probability that the organizations will have to - or. be able 
to - respond to each little change in the environment that occurs. Loose coupling 
may provide a sensitive sensing mechanism. A loosely coupled system may be a 
good system for Localized adaptation. Since some of the most important elements 
in educational organizations are teachers, classrooms, principals, and so forth, 
it may f be Consequential that in a loosely coupled system there is more room 
available for self-determination by the actors. As. a conclusion, this paper -sug- 
gests some research priorities which constitute a reasonable approach to the 
examination of loosely coupied systems. 
• 

. White, Brenda (19 79) Research: What it Means for the Practitioner , Presented at 

the Annual Meeting of the Association for Educational Communications and Techno- 
logy. (New Orleans, LA, March) 14 p. 

This paper explores ways in which practitioners can use research findings in 
. „ their daily jobs - to.keep up to date, to support program proposals, to substan- 
tiate budget requests, to improve selection of media, to' evaluate services and 
communication, and to compare methods of instruction in the use of materials. / 
Specific studies are reviewed and applications of findings are suggested. A 
bibliography is appended.. 

Wiilower, D.J. (i$6Jfc M Barriers to Change in Educational Organizations/' Theory 
into Practice , 5pS, 13-19. , a 

In this article the author focuses on the organization rather than the N 
individual, discussing some intra- dnd extra-organizational barriers to change 
in education, their sources and forms. Resistance to change develops: 1) from 
teal or perceived threat that, it poses to status; 2) if it promises to benefit' 
one^ part of the organization at the expense of another; 3) when change* ^is im- ^ 
posed from above; 4) when errors are made in introducing change. 
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Wolf, W.C., Jr. (1978) "Linking Knowledge Producers and Knowledge Users." 23 p. 

Three dif ferentTapproaches to the problem of linking educational knowledge 
producers acknowledge' users are studied in this paper. First, data are 
reported describing the knowledge generatWand knowledge communication behav- 
ior of twelve successful innovators. Second^ characteristics of twelve programs, 
conceived to train persons as change and/or linkage agents, are described in 
detail and then critiqued. And third, a ten-step dissemination methodology , 
designed to meet needs through the dissemination of products, is described. The 
paper summarizes what one might do - upon conceiving of a new practice, product, 
or idea - to determine the worth of the innovation and then to determine how 
best to inform others of its existence. I 

* • » . j ■ 

Wolf.-W.C, Jr. and A. John Fiorino (1972) "A Study of Educational Knowledge 
, Diffusion and Utilization." .Massachusetts University, Amherst, Mass.,. 125 p. * 

Some six hundred educators were studied in depth to determine their experi- 
ences with innovation, the influences of recognized diffusion agents upon their 
adoption of innovations, the characteristics of selected target audiences in 
relation -to the adoption of innovations to personal practice, and relationships 
between five distinguishable stages of innovation adoption described by rural * 
sociologists and the adoption process described by randomly selected educators. 
Among the many findings of the study were that; most of the innovative activity' 
was incidental to the operation and financing of the established order; most of 
the innovations discussed were drawn from 'outside the environ of the practitioner 
and used in tactor after modif ications; pefrsonel, direct involvement - type dif- 
fusion strateA (colleague contact, . workshops, institutes, courses) were more 
popular with ^bvative subjects. It was also found that innovators have more 
information sources and more cosmopolite sources of information than do non- 
innovators. The five-step pattern of innovation diffusion commonly seen in 
agriculture (awareness/interest/evaluation/trial/adoption) was found to be 
relatively applicable fo the field of education. 
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Worth, Walter H. (1978) "Perspectives on Policy Formation," Canadian Journal of 
Higher Education . Vol 8, No. 2, pp. 1-7. 

The relationships between policy formation and research is explored. Three 
theories accounting for the non-utilization of research, ways in which research 
is utilized by administrators, and situational .factors influencing use are 
Covered. Implications for future action within an increasingly political frame- 
work ajre presented. 

.n, R.K. and M.K. Gwaltney (1981) "Organizations Collaborating to Improve Educational 
Practice, Abt Associates, Inc. Prepared for the Research and Educational Practice 
°rogram, National Institute of Education. <y» 

' $ 
. This study was an investigation of knowledge utilization as an interorganiza- 
tional process. This study specifically focused on one particular type of inter- 
organizational arrangement - between local school districts and regional education 
agencies (REAs) . The study intensively examined three REA arrangements which were 
selected because of their successful operation. The major focus of this research 
was on how these arrangements worked successfully to provide knowledge utilization 
services. Th^ee types of services were examined; (1) staff development ; (2) 
linker assistance; and (3) information retrieval. These three* types of services 
represent the major' knowledge utilization services that can exist between two 
collaborating organizations. The findings from the study indicate that the REA's 
and their member school districts had developed an active and satisfactory arrange- 
ment ,for the knowledge utilization process to the extent that they: (1) had access - 
to external resources; (2) developed interpersonal networks; and (3) involved mu- 
tual exchanges and appropriate collaborative mandates. Because the findings are 
based on three case studies, the authors point to the need for further corrobora- 
tory research. 



Zaltman, G., Florio, D. f and L. Sikorski (1977) Dynamic Educational Change . NY: 
Free Press. 

This book is a sourcebook for planning and managing" changes within schools and 
school districts. It presents an integrated conceptual framework for change; and 
specific guidelines for< implementing- changes and solving problems that often con- 
front organizational innovators. The book begins with a discussion of the nature 
of. change in education with special' emphasis on how schools are different from 
most other types of complex organizations. Based on purrerit models and strategies 
for change, the authors propose the "Proactive/Interactive Change Model" (P/ICM) 
as a means to facilitate understanding of the stages \ in the innovation process. 
The basic purposes of this book are summarized as follows: 

1) to provide guidelines for initiating and sustaining change in a broad 
K range of educational settings 

2) to help build a capacity for self-renewal in educational organisations 

3) to serva as a primer on the management of educational change for the stu- 
dent of * educational administration 

4) to provide a structure within which much of the literature on educational 
change can be viewed * 

5) to introduce the reader to the best and most relevant work on change out- 
side the field of education -a 

6) to present an eclectic model of change. 
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III , CONTEXTS FOR CONDUCTING FIELD RESEARCH ON 
KNOWLEDGE USE AND SCHOOL IMPROVEMENT 



Problem Specification 
Past Efforts to. Alleviate this Problem 
—Consultants' Assuming Internal Roles 
—Products as Media of Knowledge Introduction 
—Knowledge Introduced Through Collaborative 

Planned Change Effects 
—Other Approaches to Knowledge Use Studies 
Events Sampling 
Phase Sampling 
Stakeholder Sampling 

Combining EventL- Phase and Stakeholders Sampling 



Evelyn Fisher 




ABSTRACT 



The purp9se of this. paper 1s to outline and 
justify an approach that permits Investiga- 
tors to sample and study key " events, "f 
"phases" and "stakeholders" as a means to 
overcoming the problem of Identifying occa- 
sions of knowledge use for study or biasing 
the research by concentrating on a single 
change attempt. 



A major problem in studying knowledge use in school policymaking and prac- 
tice for school improvement has been the elusive nature of many of the occasions 
of knowledge use. Individuals within a school district are continually being » 
exposed to or seeking knowledge which may influence their "own perceptions, 
definitions of situations and practices. Indeed, the emphasis on continuing 
education, credential ling and salary increments for graduate credits earned by 
professionals is intended to encourage individuals to acquire knowledge for 
use. Measuring the change in knowledge made accessible, possibly used and 
integrated with experiential knowledge and knowledge-in-use by individuals is 
extremely difficult. Any significant impact on the system as a whole is un- 
likely to occur, however, unless the knowledge claims that the Individual 
encounters are shared with others in some formal manner. Our concern 1s there- 
fore limited to situations in which knowledge claims are put forth and trans- 
acted in a collective setting. The introduction of knowledge by any Individual 
within the system becomes one possible instigation of a collective knowledge 
transaction. 

Even if the phenomena for study are restricted to knowledge transactions 
and use that potentially affect collective learning capacities, identifying 

4 

occasions for empirical research requires careful consideration. 

This working paper (1) specifies this problem of identifying occasions 
of knowledge use for research; (2) reviews and evaluabs past research efforts • 
to alleviate 1t; and (3), outlines and justifies an approach that permits in- 
vestigators to' sample and study key "events," "phases", and "stakeholders." 
T) Problem Specification 

Researchers cannot themselves become aware of all of the possible instances 
in -which knowledge enters an on-going system of actors and activities. If the 
-researchers intend to stufly various situations of knowledge use they must have 
the assistance of persons within the system of action to alert them to events 



b; 



that may be studied. Yet In order to provide this notification, even the most 
cooperative and conscientious informants require criteria for defining a situ- 
ation as constituting an Instance that provides a potential Opportunity for 
studying knowledge ,use. While the concern 1s with collective knowledge trans- 
acting with symbolically mediated and pubWly shared meanings, there must be 
a medium or channel that introduces knowledge 1n some form or a" situation 
calling forth knowledge' claims that produces an occasion for discussion among 
members of a collectivity. This means then that by the time that'ev'en the 
most sensitive observer is aware that knowledge transacting is occurring, 
there will be a prehistory as well as possibly consequences that may be studied. 

From an analytic perspective, the introductory of knowledge, the putting 
forth and contesting of knowledge claims that occur' and the consequences 
which may ensue may be treated as an entity. This has characterized most case 
studies of knowledge use or school change. It must be recognized, however, ' 
that this 1s an Imposition of ent1t1v1ty and is not necessarily experienced 
as such by the actors Involved. For the actors, these occurrences are part 
of the natural flow of activities of daily life in a school district. The 
process that the researcher goes through of selecting specific Interactions 
and actions may be likened ^o writing a play in which the author truncates 
time eJapslng. between significant events and eliminates the extraneous hap- 
penings that would be assumed to occur in the natural course of everyday life 
but would not Influence the theme's development. So the researcher selects 
from the myriad of social interactions and actions that do go on those that 
are thematlcally linked to the problem as defined by the researcher. These 
may be observed and documented, or reconstructed. They are analyzed with the 
use of concepts, constructs .and research techniques to explicate the knowledge 

use related processes and influences on these processes. , 

«. - 

The problem, then, is to become aware when knowledge entering into the 



system of on-going activities is put forth or subjected to collective trans- 
actions which could potentially affect the system' itself or some- component of 
the activities or practices that are engaged in.v 
'2) Past Efforts To.Alleviate This Problem 

'- In reviewing past research on knowledge use to effect improvement in 
educational practices, it becomei«evident that many of the studies were re- 
stricted to purposive interventions- into, an on-going system by outside agents 
or agencies. The problem of identifying situations or occasions pf knowledge 
use was thus handled by limiting the range of studies to known attempts to 
introduce change into the system. Relatively 1 ittle attention was given to 
occasions instigated by persons within the system or those influenced by other 
systems as, for example, the legal system, except as these constituted impinge- 
ments on the course of events under- study. In some instances these were 
considered as "derailing" intentions or even causing |ttempts at planned change 
to be aborted. v , 

The research literature Is' rather heavily biased toward, an imagery of 

schools as conservative environments in which the organizational cTimate or 

structure and the practices are in need, oY change. The research then focuses 

attention on attempts to effect change. There have been numerous approaches' 

to and strategies for attempting to alter school" practices. The emphasis has 

been on d«velo^ing models, strategies principles of change so that knowledge 

can be introduced into a system and the Jjractiti oners encouraged to use it. 

The implication is necessarily that the knowledge-in-use within an existing ' ' 

system of practices is deficient and needs to be altered or . replaced by some 

A ' . 

knowledge produced elsewhere and available for adoption. Although our own 

;program of research will take a significant departure from this research, the 

cumulated' literature can be interrogated -with respect to how the problem of 

knowledge use has, been defined and subjected to study. 

j • 
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In conducting this review, a classification schema has been developed in 
order to Impose some order. It should be understood as a means to creating ' 
order rather than description of sequential developments. Even though some 
planned change action systems evolved as attempts to Improve upon previous 
ones and some research on these action systems have responded to new problem 
formulations and systematic attention to newly explicated Issues and concerns, 
there was not a .ynllaHty 1n the efforts that ex post facto analyses may seem 

to Impose:- 1 ' r 

\ 

(a) Consultants Assuming Internal Roles 

> 

' Knowledge has often been introd/ced into a system by external consultants 
'These were often faculty members froln institutes of hlpier education working 
with school districts to effect / improvement. These consultants either had 

preestablished plans regarding desired changes and strategies for effecting 

- ' / f 
them or they went Into the system tb identify problems and t^ their knowledge 

and expertise for reviving them. In either case, the consultants or' change 
ag^fits became the medium through which knowledge was Introduced. They tem- 
porarily took on TeadfersMp positions in which they identified or formujated 
the problems, decided what strategies or changes could be Introduced to solve 
them and tapped their own knowledge and expertise. Once they had effected the 
changes by getting people to change their practices or procedures^ 1t was 
assumed that the regular participants in the system would sustain the changes 
that had been made. The consultants then left the district. Frequeftb^bpoks, 
reports or papers were written by these persons retrospectively in which they 
reflected on their successes or the obstacles and impediments to knowledge use 
of social change that they encountered. 

Characteristic of external change agents who attended to strategies for 
entering a system and contacting classes or categories of- role incumbents in 
order to effect change is the work of Fantini and Weinstein (1966). Their 
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J self study focuses attention on strategies they used for involving or coop- 
ting strategic groups in order to create change in the complex bureaucracy of 
schooling. .These authors were striving to develop a model for how to effect 
change in school districts' that could be used by other external consultants. 
-^Jhis approach did not increase the capacities within districts to^ Identify 
needs and effect their own improvements but rather cYeated at least temporary 
dependencies on these external consultants who had assumed a leadership func- 
tion. Experience with -this approach often later revealed that once the exter- 

4 

* 

nal change agents left the system, the changes were not necessarily sustained: 
• This seemed to be particularly the case 1f dissatisfactions or difficulties 
were encountered with the "new way of doing things." 'Since the cons^Uaits 
were no longer available to turn to for help and the social support' system or 
personal Incentives were lacking for 'solving any problems, it seems that old 
practices weire reinstated (Plncus, 19 ). k 

Numerous studies concentrated on how to bring about organizational change 
(Baldrldge and Deal, 1975). Although some of these change attempts Involved, 
at least 1n part, the introduction of currlcular change or instructional 
Innovations, the focus remained at the level of "Inducing change." Involvement 
by actors within the system was attended to at the Jevel of participation, not 
at the level of the cognitive processes such as comprehension, assessment or 
knowledge use. Research publications usually did not give detalTs of the 
changes themselves or the knowledge bases but focused attention on the "degree^ 
.-of Implementation" or "obstacles to change" (Gross, Glacquinta and Bernstein, 
1971). " 

'(b) Products as Media of Knowledge Introduction 

During the 1960's, there was a widely held view that educational outcomes 
could ijie Improved 1f the theories and research findings from the behavioral 
and social sciences were applied" to educational practices (Cronback and Su^pe, 
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1969; Derschlmer, 1976). With the establishment of federally-supported . 
research and development centers and regional laboratories, an emphasis was 
placed on developing innovative programs that could be disseminated to poten- 
tial users. Concomitant educational programs such as Head Start, Follow 
Through, Right to Read, etc. encouraged the adoption of educational Innova- 
tions. Competing with, though not completely replacing, the single or small 
group of consul tants*as mediators of eha/ge approach was this concept of 
"packageable products." 

Even when Innovations were developed in collaboration with school per- 
sonnel from one district or single school, the expectation was that there 
would be a "finished product" that would be appropriate for use 1rr other 
schools. The Instructional technology was often developed Initlally^withln 
the context of the research and development organization and u^ed the find- 
1ngs of research, on learning and learning environments conducted from a 
ps ^ o1ogica1 Perspective (Gl'aser and Resnick, 1972). This knowledge was 
integrated or synthesized with craft and experiential knowledge 1n the 
development of Innovative programs'." The program was tried out experimen- 
tally/In a school and formati-vely evaluated and revised as necessary (BIckeK 
197$. 

Further dissemination and assistance with Implementation became tasks 
for others once the product was considered to be completed by the developers, 
althdugh developers sometimes served as consultants. This led to an elabor- 
ated divlslori^of labor, I"t was the basis for the establishment of federally- 
funded reg16nal laboratories whose tasks were to help schools 1n their region 
select and Implement appropriate educational innovations. This Initial plan 
was soon modified for a number of reason^. -Some of the^ Innovations were so 
complex that once a laboratory staff had developed strategies and training 
programs for helping school district personnel with the introduction of the 



Innovation, 1t seemed more efficient to "specialize" 1n that Innovation nat- 
ionally wherever 1t was adopted. A second reason was that there were insuffi- 
cient "completed products" to select from among that were appropriate for 
schools 1n the region. A thirchwas that many innovations were perceived by 
the laboratory staff as incomplete or in need of modifications as they Intro- 
duced them Into schools. Many regional^ laboratories became Involved in their 
own de#Popment activities. In addition to those who designed and developed 
Innovations, there were those who designed and .developed systems for dissemi- 
nating Information on available jpnovations (see, for example, Radnor, et al_, 
1976), Unking systems with change agents or change agencies (see, for exam- 
ple, Havelock, 1969) and storage and retrieval systems such- as ERIC at the 
national level or RISE 1n Pennsylvania at the state level. Those researchers 
concerned with knowledge use 1n systems of ^practice studied the functions 
fulfilled by these systems, developed models of knowledge production- and use 
sys^fems, studied the roles involved, (Havelock and Havelock, 1973) 1n order to 
train role encumbents or conducted research on the use and users of these 
systems (Sleber, 1968; Sleber, et al_, 1972). 

It was generally assumed that 1f educational innovations were adopted 
for use, the scientific knowl'edge used by the developer was itself then being 
used. It was also assumed that innovations that were adopted were being used 
and therefore student outcomes were an evaluative measure of the* Innovation 
itself. Evaluations of change (efforts became broadly mandated. Fullan and 
Pomfret (1977) pointed out the lack of attention to what happened between 
Inflation adoption and outcome evaluation. Other researchers began to focus- 
on this Issue and tried to determine the degree to which an Innovation was 
being Implemented (Lelnhardt, 1977). This required that what went on In 
classrooms be compared with what the developer had intended although these 
Intentions were not always made explicit, since there can be an almost 



Infinite amount of variation 1n what occurs 1n a classroom at the concrete 
level, attempts were made to measure classroom processes and to relate these 
to outcome measures (Cooley and Leinhardt, 1975), This research did not 
address the Issue of how this variation among lisers came about, nor indeed 
how the classroom processes were influenced by "the innovation. 

Because knowledge use wa V s still being equated with innovation use, the 
direction of research was toward "Implementation." Many of these innovations 
were comprehensive instructional programs that not only included curricular 
materials but pedogogical strategies and techniques, recommended managemeht 
and achievement mon1toring\§ystems and practices for the tfeacher which often 
required redefinitions of the role of the teacher. Some research focused on 
organizational variables in an attempt to explain implementation variation. 
This was particularly ,the case with lairge scale surveys. The extensive Rand 
study on Implementation included both/ surveys and case studies. The latter 
revealed variation 1n use of the same innovation. Greenwood, Mann and Mc- 
Laughlin (1975) offered the exffranation that users "adapted" innovations to 
accommodate the particular characteristics of their school of classroom. 
This implied that users had a conception of how an Innovation was Intended 
fQr use, knowledge of their classroom or school and what would have to be 
altered or adapted In order to make the innovation suit that particular set- 
ting. In the process, mutual adaptation resulted in which both existing 
practices and existing innovations were altered. Fisher, Penol and Wesley 
(1979) found no evidence that such tailorirtg to the situation of use was 
occurring since variations among classrooms were just as great as the varia- 
tion among schools serving different areas/of the district. They suggested 
that teachers as users had differing Intentions and expectations so that they 
actually imputed different meaning and potential to the Innovation's compon- 
ents or management system. Magoon (1977) 1n a conceptual paper supported 



this constructs vi st explanation for variation 1n the use of Innovations. 

None of these studies on the use of innovations dealt with the phenomena 

and processes at the level of knowledge glalms. The developers of innovations 

presumably assessed scientific knowledge claims but their "products" reflected 

a great pny decisions in which scientific knowledge claims', experiential 

* • • • - 

knowledge and craft knowledge were interwoven. The practitioners used the 

innovation. While conducting preservice workshops consultants in introducing 
the innovation often gave rationales for the components or the management sy- 
stem in addition to instructions on how to use the program materials. Even 
when the rationales were not questioned or challenged by teachers, teachers 
seemed to impute tjieir own, sometimes conflicting, meaning to them* For 
example, it was frequently found that curricular components that allowed stu- 
dents to practice skills* 1n a game format or that relaxed the* Imposed struc- 
ture to allow children to explore were defined by many teachers as "play." 
"Play" was a privilege that could be earned or an opportunity that could be 
denied as a disciplinary measure. These activities were not treated as an 
integral part of a learning environment. These seemed to be no indication 
that teachers were deliberately misusing an Innovation but rather that they 
imputed different meanings to the components of the innovative program and 
used them accordingly. Other qualitative research concerned with the use of 
innovations recognized thB^ibjective meanlngs^nd attitudes (Hall, et al_, 
197S; Hall, et ai, 1979). 

(c) Knowledge Introduced Through Collaborative Planned Change Efforts 

Some attempts at Improvement 1n practices have involved a collaborative 
effort 1n which school personnel and external consultants have Identified 
problems or needs and planned changes. These have differed from the cases 
where the external consultant took over the leadership role. While these 
opportunities to study knowledge use could readily be identified, since 



consultants were entering a school district, the resulting literature has 
focused on explicating "principles of change" or "strategies of change" 
(Rothman, 19/4; Zaltman and Duncan, 1977). These have provided prescriptive 
guidelines for how to effect change distilled from the experiences of the 
persons who have served in a consulting role or comparative research on plan- 
ned change efforts^—- < * 
(d) Other Approaches to Knowledge Use Studies 

• There are three additional types of studies that have been conducted 
that have not been restricted in their focus to specific intentional efforts 
from outside agents or agencies. The first type may be classified as diffu- 
sion studies (Rogers, '1962; Rogers and Shoemaker, 1971) where tbe interest 
has been on the adoption of ideas or innovations. The difficulty encountered 
in studying knowledge use processes in cases of" diffusion of knowledge is 
that they were rarely studied concomitantly but relied heavily on reconstruc- 
tions by the participants or inferences made by the researcher. 

A second type attempted to i den tify icriowl edge changes that have occurred 
over time as a result of widespread acceptance of scientific knowledge such 
that it. became part of the diffuse culture. GetzeTs (1978), for example, 
suggested that the physical arrangement of classrooms in various eras reflected 
different perceptions of the learner and the learning process. Although this 
does not provide insights into the knowledge use processes, it does call 
attention to the diffuse culture which may influence knowledge-in-u'se or the 
more global reality constructions within which knowledge-related events are 
experienced and knowledge claims assessed. 

The third category focuses attention on knowledge-in-use rather than 
specifically on new knowledge entering into systems of practice. These stu- 
dies are usually qualitative, and attend to- the ways in which people make 
sense of their everyday life situations. * 
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2) The Knowledge Use Approach to School Change 

These solutions to identifying occasions of knowledge use that are re- 
flected in the literature are too limiting for our purposes. They either 
concentrate on a single, deliberate attempt to change school practices, some 
specific sector or characteristics of an information flow system or on know- 
ledge-in-use. While these studies have made contributions to our understand- 
ing of schools and attempts to improve school practices, they have not 
i adequately captured the school as a functioning community of practitioners 
into which knowledge claims are introduced, assessed and used. 

n 

EVENTS SAMPLING 

It is our intention, to begin with the school district as a formally or- 
ganized, knowledge using community of practice. Our concern is with knowledge 

claims that may be introduced or situations that may occur that instigate the 

- / 

putting forth and contesting of knowledge claims. The events which precipi- 
tate collective knowledge transacting may vary condiderably. In order to 
study knowledge use processes and issues that may affect practices, we do not 
want to limit research to any particular change attempt but rather to sample 
the variety of events that do occur. 

Although events that do occur must be classified in order to select those 
$hat differ maximally according to some criteria, it is necessary to generate 
a schema in advance as a guideline for informants. This schema is meant as 
a sensitizing heuristic and not as an exhaustive framework of all logical 
possibilities. 
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EVENT GENERATED FROM WITHIN 



Precipitating 
conditions 

(1) Problem 

Identification 

(.2) Need Assessment 

(3) Alternative 

Envisioned 

(4) Situation Altered 

or Redefined 



B INST I GATOR 



-COMMUNICATIVE" MODE 



TABLE I 

Discussion of categories: 

A-(l) An event that precipitates collective knowledge transacting may be the 
identification and articulation of a problem as defined by an individual 
or group within the system. 

B-(l) The definition of some situation or condition as problematic may be art- 
iculated by an individual in a position of authority and responsibility 
who can impose that definition as a basis for problem-solving action. 

The definition of some situation or condition as problematic may be 
articulated by an individual in a position of responsibility for some 
subset of tasks where problem-solving activity may require sanctioning 
and/or cooperation by others. 

The definition of some situation or condition as problematic may be 
articulated- by a subset of persons who share similar sets of tasks-and 
responsibilities. ;' 

A-(2) An event that precipitates collective knowledge transacting may be the 
assessment of a need that is unfulfilled or inadequately fulfilled. 

B-(2) The need may pertain to the system as a whole and be assessed by an 
individual from the perspective. 

The need may pertain to a sub-category of tasks or subgroup of actors 
and be assessed by an individual or individuals from the perspective of 
the sub-category. 

The need may pertain to a, sub-category of tasks* or- subgroup of actors 
and be assessed by an individual or individuals from the perspective of 
within the system but outside the subgroup. 

A-(3) An alternative way of' accomplishing some tasks or attaining some desired 
-outcomes may be envisioned and precipitate collective knowledge trans- 
acting. - 
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B-(3) An alternative way of more adequately, effectively or efficiently accom- 
plishing a -desired outcome or improving a situation or condition may be 
envisioned by an individual in a position of authority to mandate change. 

■ An alternative way of more adequately, effectively or efficiently accom- 
plishing a desired outcome or improving a situation or condition may be 
envisioned by an individual in a position of responsibility for the 
efforts of others in some task domain who recommends change. 

An alternative way of more adequately, effectively or efficiently accom- 
plishing a desired outcome or improving a situation or condition may be 
envisioned by an individual who requests the sanctioning^ support of 
others to effect change. 

A-(4) A situation which is altered or redefined may precipitate knowledge trans- 
acting 1n order to accommodate existing practices to this change. 

The definition of a situation as having changed or in the process of 
changing may be put forth by any individual or collectivity. 

C The communicative mode may be a written directive, an in-service work- 
shop, meeting, formation of a task force or committee. 



EVENT GENERATED FROM OUTSIDE 



(1) FEDERAL 

(2) STATE 

(3) LOCAL 

(4) OTHER 



(A). AUTHORITATIVE 
Law 

Legal Decision 
Regulatory Mandate 



Law 

Legal Decision 
Regulatory Mandate 



School Board Deci 
slon 

Union Contract- 



(B) ADVOCATIVE 

Federal aid 
for Proposed 
Projects Within • 
Policy Guidelines 

State aid for 
Proposed Project 
Within Policy 
Guidelines 

Demonstrations 



Political Inter- 
est Groups 
Conmercial Agents 



(C) FACILITATIVE 

Federal aid 
to Meet a Need 



Technical 
Assistance 
and Training 



In-service 
Workshops 

Technical Assis- 
tance 

Conmercial Traia- 
ing 



TABLE II 
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(1) A - Events Initiated outside the school district may impact on the district 

Instigating collective knowledge transacting to accommodate practices 
to requirements binding on the district. The passage of federal law or 
a legal decision establishing action 1n conformity with interpretation 
of a law may be such an event. Federal policy' mandates affecting cur- 
rent practices may be another such event. 

B - Announcement of or change in federally-funded or aided programs for ' 
qualified or approved projects may be an event Instigating knowledge 
transactions within a school district. These are advocative rather 
than authoritative s'lnce districts need not apply for federal program 
participation. 

C - Federal aid to facilitate a district's meeting of a need (such as com- 
pensatory education) may be an event that precipitates knowledge trans- 
acting. 

(2) A - Changes 1n laws, legal decisions or regulatory mandates issued from the 

.state may be events that Instigate knowledge transacting since school 
districts are required to comply. 

B - The state may provide financial aid to school districts to assist them 
1n undertaking some advocated activity. This may Instigate knowledge 
transacting. 

C - The state may provide technical assistance and training 1n order to 
promote some particular practices. 

(3) A - A School Board decision may be an event that initiates knowledge trans- 

acting as decisions are translated into action plans. 

B - Demonstrations of how other schools or classrooms are operating may 
instigate knowledge transacting. 

C - Inservlce workshops with consultants from outside the district may be 
an event Instigating collective knowledge transactions. 

(4) A - Union contracts, their formulation, negotiation and Implementation may 

be events that instigate knowledge transactions. 

B - Political Interest Groups may bring pressure on school districts to 
add or alter practices. These may lead to collective knowledge trans- 
actions. h * 

Commercial agents may attempt to sell a product and this may Instigate 
collective knowledge transacting. 

C - Technical assistance provided by persons outside the school district 
1n the public br private sector may facilitate or Instigate collective 
knowledge transacting. 
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PHASE SAMPLING , \ 

At any time that research is conducted to explicate respondents' assess- 
ments or use of knowledge associated with any event, different phases may be 
occurring. Tentatively such phases may be considered exposure to knowledge 
claim, assessment of knowledge claim as a collective transaction, integration 
of knowledge with knowledge-in-use, reinforcement or reconsideration of know- 
ledge claim, use or rejection of knowledge claim or Institutionalization of 
knowledge at which point it has become knowledge-in-use. Although some empir- 
ical research by Hall, et ai (1975) has led to categories of "levels of use," 
little research has attended to.the influences on these processes, or what 
causes variation withtn a community of practitioners. By combining the sam- 
pling of phases with network mapping and mul tiattribute analysis, a more 
precise explanation can be generated. 

STAKEHOLDER SAMPLING 

Individuals as subgroups may have different knowledge-in-use, frames of 
reference, different assumptions or definitions of situations that influence 
their problem structuring, assessment and use of knowledge. In order to 
ensure that a balanced range of possible knowledge transactions and assessments 
are included in a study of knowledge use within a school district, it is criti- 
cal to use a sample of stakeholders that will maximize differences. 

A general 11st of stakeholders in education can be generated to include 
categories of persons in particular role relationships to the systpm. These 
would include categories such as teachers, students, various administrative 
personnel, school board members, union representatives, parents. To some ex- 
tent the sampling would have to be related to the events being used since not 
all events would affect all stakeholders directly enough for them to be 
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informed. It is also the case that events may be very differently perceived, 
defined and reacted to by members of any single stakeholder group. For this • 
reason, sampling would Involve representation of stakeholder groups and within- 
each category, sampling of persons with differences that are known among the 
stakeholder group or others as a first approximation to maximizing differences. 
It is a first approximation because it is limited to publicly shared differ- 
ences. As the research reveals bases for these differences, more refined 
sampling can be undertaken to reflect the dimensions of differentiation. 
Combining Event, Phase and Stakeholder Sampling 

By combining these three types of sampling research can be conducted 1n 
a more rigorous, focused and less obtrusive manner than the traditional case ' 
or field study. The sampling strategies allow a wide range of differences to 
be captured with minimal intrusion into the activities and schedules of parti- 
cipants. These differences may be with respect to events, to phases or to 
stakeholder frames of reference and meaning structures, all of which wiH be 
reflected 1n the selection of persons as participants in the research. Re- 
peated rounds of interviews and instrument administration will permit the ^ 
traces and bases' of any chajiges to be explicated. 
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ABSTRACT 



The purpose of this paper is to critically 
review the theoretical perspectives and 
methodological techniques of qualitative 
research. The past use and applicability 
of qualitative procedures to knbvledge 
use and school Improvement is explored. 
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I. Qualitative Research 

The term "qualitative research" is itself an smalg^n, designating a 
variety of both theoretical perspectives and methodological techniques. Among 
the former are included phenomenological sociology, existential sociology, 
cognitive sociology, reality construction, symbolic iht^raceionism, and 
ethnomethodology. Qualitative methodological procedure^ include such comnon 
techniques as participant observation and ethnography, interviews, Ufa 
histories, and the use of assorted unobtrusive measures as well as more 
uncommon techniques particularly associated with ethnomethodology such as 
breaching. 

t 

Yet while qualitative sociology Is marked by such diversity in theoretical 

perspective and methodological procedure, it does have a central animus 

> 

comprised of distinct goals and assumptions which serves as a point of 
coalescence. Qualitative sociology takes as its object the reality of 
everyday Ufa which is regarded as an activaly Constructed product of human 
agency. The order inhering in such reality if/not regarded as independent of 
the actors 1 purposes, intentions or motives. Rathet, order emergent in social 
interaction processes is taken to be the product of meaningful hlmMn activity. 
That is to say that actors Impute meaning to their own and to others 1 actions 
and that such meanings are the sine qua not* of social action. As Douglas 
(1970:lx) notes, "the 'forces' that move human beings, as human^beings rather 
than simply as human bodies... are 'meanir«ful stuff.' They are internal ideas, 
feelings v and motiyes." 



> 



It follows then that tp understand social action and the reality . ' 
constructed by the meaningful activity of associated actors one has to go 
directly to the actor, to understand behavior from the actor's point of view - 
that Is Interpretive understanding or what Max Weber called "verstehen." To 

this end "qualitative 11 methods yield generally descriptive data that enable 
the researcher to see the world as, subjects see It (Bogdan and Taylor, 1975:2). 

. This review of elements of qualitative sociology's central animus has 
been Intentionally kept brief so that It might simply servers an Introduction •• 
to the more specific treatment of theoretical perspectives and methodological 
techniques which follows. For a fuller discussion of qualitative sociology 
with an emphasis on both theoretical and methodological issues see especially 
fiogdan and Taylor (1975), Cicourel (1964 and 1968), Denzin (1970 and 1970a), 
and Schwartz and Jacobs (1979). 

A. Selected Theoretical Perspectives' 

- „ f • r • ■ , ■ . 

Chief ampng the theoretical perspectives comprising qualitative sociology 
is the phenomenological perspective based upon th^ phenomeno log leal philosophies 
of Alfred Schutz and Edmund Husserl, and the earlier verstehen sociology of 
Max Weber. Phenomenological Inquiry seeks to get at the things themselves, 
thaf:' is to see the phenomena as directly as possible or as they are presented 
in immediate experience. Since the knowing subject 'always experiences things 
first hand a person as such a subject and also as a sociological observer has 
the opportunity tor directly examine that experience. 

Consistent with Husserl, phfnomenologistsdistinquish between the natural 
and the theoretic stance taken toward the everyday life world. The natural 
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stance: 

...is the stance in which the everyday world is taken for granted «, 
as it is experienced in everyday life... It is a stance that does - 
not raise serious questions concerningstfie nature of the everyday 
experience but, instead, takes that experience as a fact 
(Douglas, 1970:14). 

In contrast, the theoretic stance toward the everyday world involves 

standing apart from and reflecting upon the experience taken for granted in 

the natural stance. The theoretic stance emerges from a particular activity 

of the observing subject which Husserl referred to as phenomenological 

suspension or "epoche." Employing this procedure, the phenomenological 

observer /researcher "brackets" the assumptions which structure the experience 

of actors in their everyday life worlds. In this way > clearer vision of 

everyday life emerges from a fuller examination of the myriad of elements 

constituting everyday experience. As Psathas (1973:15) observes: 

...in a phenomenological approach, a whole range of experiences 
that would ordinarily be ruled out are instead, as objects of 
intentional consciousness, entitled to the same serious attention 
and study., Thus bracketing enables one (to. expand tiis view and 
embrace more aspect* of the world for study. 

Finally it is important to note that commitment by phenomenological 
researchers to the theoretic stance results in certain methodological 
implications. The phenomena of everyday life must be studied on their own 
terms and only those methods' of observation and analytic modes that retain 
the integrity of the phenomena should be employed (Douglas, 1970:16). 

A related theoretical perspective associated with qualitative sociology 
is symbolic interactionism which traces its roots to the works of John Dewey, 
Charles Horton Cooley, Robert E. Park, W.I. Thomas, and George Herbert Mead. 
Central to symbolic interactionism is the notion of the actor's im^bsition of 

^ i 



meaning upon the action situations in which he or she is Implicated in the form 
of interpretations and definitions of elements in those action situations. 

Thus situations are rendered meaningful only as the result of the active 

* » 

agency of human actors. In situated interaction processes Actors develop 
definitions of the situation in concert with other actors engaged therein and 
they act_in terms of these definitions." Order inhering in interaction' 
processes is taken to-be an emergent property of the interaction arising out- 

m 

of the negotiation of convergent definitions of the situation by associated 
actors. 

A most succinct statement on the character of symbolic interactionism is 
providfd by one of its most influential contemporary proponents, Herbert 
Blumer (1956:686): 

r 4 

We can...lo6k upon human life as chiefly a "vast interpretative V 
process in whfch people, singly and collectively, guide themselves 
by defining the objects, events, and situations which t$h*y 
encounter — And scheme designed to analyze human grpup *lif « in its 
general* character has to fit this process of interpretation. 

'As is the cMe for all theoretical .perspectives, distinct methodological 

considerations follow upon the use of the Symbolic interactionist approach. 

As Denzin (1970) suggests, at least two methodological principles may be 

identified. In the first place, an adequate investigation from the point of 

view of the symbolic interactionist perspective requires that both symbols 

and interaction be addressed. "To focus only on symbols. . .fails to record 

the emergent and novel relationships these symbols have with observable 

behavior" (Denzin, 1970:7-8). Secondly, the investigator must attend to 

shifting self-definitions in interaction processes. He must- take the xole of 

the other whom he is studying in order to assess the meaning the situation has 



for the actor from his or her own point of view. 

A third and more recently articulatfed theoretical perspective associated 
* irtth qualitative sociology is ethnomethodology. Numbered among its most 
influential proponents are Harold Garfinkel, Jack Douglas, Egon Bittner, 
Don Zimmerman, and D. Lawrence Wieder. The term "ethnomethodology" itself 
does not refer to methods as such but rather to the qbject of inquiry. That 
object of inquiry is the practical ways, procedures, or methods by which people 
order their everyday life situations. The task of the ethnomethodologist 
"...is to examine the ways people apply abstract rules and conmonaense 
understandings in situations in order to make actions appear routine, explicable, 

s. 

and unambiguous" (Bogdan and Taylor, 1975:16). These understandings or 
meanings are taken to be practical acccbplishments of thje actors. Likewise, 
as Garfinkel (1967:viii) suggests, ethnomethodology has as its aim the analysis 
of, "...the formal properties of common sense activities as a practical organi- 

/ V > 

zational accomplishment. 

s 

A^ong the formal properties of the common sense activities of everyday 
life of greatest interest to the ethnomethodologist are reflexivity, inten- 
tionality, and indexicality . In analyzing what Garfinkel calls the 
"ref lexivities of accounts" he notes^that actors 1 , accounts of reality and 
its workings are, constituent parts of that reality they describe. This is the 
central point which emerges in ethnomethodological studies of conversations 
as verbal accounts in the wo.rks of Cicourel and Sacks and of wfitten %c\rds 
in organizational settings such as social welfare agencies (Zimnerman, 1970), 
Juvenile probation agencies (Cicourel, 1968), and mental health clinics 
(Garfinkel, 1967). 
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A related property is that of intentlonality of thought and action 
which has its origins in Schutz's principle that interests organize our 
> constructs in th* common sense world (Heeren, 1970:45),. As Douglas (1970:26) 
noVes, "information and ideas become conscious as we see them as useful to the 
attainment of our purposes at hand." 

Indexicality refers to the fact that rules, accounts, or oth*r meaningful i 
components of practical activities can be understood only contex^ually. That 
is to say that their meaning is contingent upon the broader context in which 
they occur. For example rules in organizations can only be understood as 
"...practical problems that themselves arise out of the association of „those 
people, facing those problems, in those periods of . time" (Manning, 1970:242> 
see also Garfinkel, 1967: chap. 1; and, Bar Hillel, 1954). 

Methodologically, the ethnomethodological perspective has become 
associated with the use of a number of uncommon techniques, purposefully 
highly obtrusive in character, involving the breaking of rules of eyeryday 
life in interaction situations to render them conscious and thereby 
analyzable. For a review of these techniques see Garfinkel (1967), Mehan an<f 
Wood (1975), and Schwartz and Jacobs (1979) . 

Together then these three dominant theoretical perspectives of qualitative 
sociology signal the emergence of interest in the reality of everyday practical 
activity. A reality dynamically constructed by the creative agency of the ^ 
human actor, and a reality which requires, as a condition of its illumination, 
the use by sociological investigators of particular methodolgical procedures 
pertinent to it. A 
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B. Selected Methodological Techniques 

Any methodology may be characterized by the particular type of data 
which it produces. Qualitative methodologies produce descriptive data in 
the form. of subjects' spoken or written words and observable behavfor. 
According to Patton (1980:22) qualitative methods generate raw data from the 
empirical ^ior Id consisting of; 

detailed descriptions of situations, events, people, interactions, 
f and observed behaviors; direct quotations from people about their 
experiences, attitudes, beliefs, and thoughts; and excerpts or 
entire passages from documents, correspondence* records, and case 
histories. " 

Unlike' data generated through conventional quantitative procedures, qualitative 
data are marked by two distinguishing features, depth and detail. These two 
features serve as the intrinsic source of value of qualitative data. Addition- 
ally, as compared to more conventional quantitative procedures, qualitative 
procedures have a more "ad hoc" quality. This is not to say that qualita- 
tive researchers are more haphazard in formulating or carrying out such pro- 
cedures relative to their quantitative counterparts. Rather, they are 
simply acutely aware of the tremendous variety and diversity of concrete action 
phenomena and they let the problem situation determine the method,. Indeed, 
among qualitative researchers this procedural guideline is elevated to a 
cardinal tenet. Mills (1959:224) states the case thusly: "Avoid fetishism of 
method and technique. . .let every man be his own methodologist." Methodological 
orthodoxy is thus considered anathema to "good" qualitative research. And 
qualitative researchers are much freer to innovate, to experiment, to devise 
de novo techniques with %hich to uncover the often seemingly recalcitrant 
reality of everyday life. 
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Thus the assessment of qualitative procedures is inherently fraught 
with problems given these considerations and such an assessment can never lay 
claim to exhauativeness as a quality. Yet as a way of proceeding perhaps it 
would be nonetheleas appropriate to very briefly review more conventional 
qualitative techniques in an attempt to aensitize the reader to problems 
associated with qualitative research generally. 

J 

Chief among the more conventional qualitative techniques frequently 
employed in social research efforts is participant observation which represents 
a commitment on the P»rt of the investigator to intimately participate in 
the experiences of the subjects under study. Participant observation is 
particularly suited to permit the investigator to experience the world of 
his/her subjects from their point of view. Properly conceived, participant 
observation entails the simultaneous use of multiple methods, becoming a 
"triangulated methodology" (Denzin, 1970a:366). The participant observer 
collects and analyzes documents, conducts interviews with informants and 
participates directly in subjects' activities. 

Participant observation may assume a variety of forma depending upon 
the particular role taken by the investigator. Building upon the earlier 
work of Junker (1952), Gold (in Denzin, 1970a: 370-380) has developed a four- 
fold typology of observation roles to include: complete participation, 
participant-as-observer, observer-as-participant, and complete observer. 
Complete participation characterizes the activity of the covert participant 
observer who never reveals his/her identity to the subjects under study. He/she 
assumes the role of member. The participant-as-observer represents the most 
commonly used observational method or overt participant observation where the 
researcher makes his/her identity known but attempts to "normally" participate 
in subjects' activities. The observer-as-participant role ia best represented 



in surveys where Che interviewer may see his/her respondents only on.ce in « 
brief encounter. Finally, the complete observer is "best seen in the experiment 
where the observer does not participate at all in the events being studied. 

A distinct set of problems is associated with each of these observational 

r 

roles. For these see: in Denzin, 1970a; Orne, Leik, Riecken, and Olesen and 
Whittaker; and, Polsky (1967), and Schwartz and Schwartz (1955). But for here 
suffice it to say that as one approaches the role of complete participant, 
avoidance of "going native" or problems of maintaining objectivity are 
exacerbated while, as one approaches the role of complete observer, descriptive 
detail of data and deeper theoretical insight are forsaken. 

For an excellent treatment of participant observation see: Berreman, 
1962; Bruyn, 1966; McCall and Simmons, 1969; Adams and Preiss, 1960; Schatzman 
and Strauss, 1973; and many of the essays in Becker, 1970. 

A second commonly employed qualitative procedure which is the most 
basic source of sociological data is the interview which is simply a face-to- 
f)ace conversational exchange in which one person elicits information from 
another. There are different types of interviews ranging from highly structured 
to highly unstructured. A .structured interview consists of a single set of 
similarly ordered questions given to all sample respondents. Structured 
interviews result in the elicitation of common information from all respondents 
which enhances coding capabilities yet it assumes uniform understanding of 
questions by all respondents. Such en assumption is often unfounded (Benney 
and Hughes in Denzin, 1970a; 190-198) . The structured interview is best 
Suited to hypothesis testing and rigorous qualification of results (Maccoby 
and Maccoby, 1954). 

The unstructured interview employs neither fixed questions nor a 
predetermined order for asking them. This^method is more fluid and is best 
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suited for probing subjects' 111-def lned attitudes or In situations where • 
diversity of opinions Is prevalent. Problems associated with unstructured 
Interviewing include difficulty in coding and lower degrees of repllcablllty. 
This type la best suited for exploratory studies (Maccoby and Maccoby, 1954). 

Generally Interviewing Is a good source of data on subjects' perspectives 
or views on particular topics but It suffers from a number of problems 
peculiar to It as a type of communicative Interactional encounter. The . 
encounter Is productive of a number of emergent properties, each of which 
may jeopardize the validity of the Information revealed. Tone of voice, 
dress, mood, vocabulary, a variety of status dlff erentlals, length of Interview 
all of these features Impact upon Interview lnteractants, determining the 
meanings lnteractants attach to the situation as well as to the Information 
transmitted and received therein. According to Denzln (1970; 132-141) , sources 
of Invalidity In the Interview because of Its peculiar character as a 
communicative encounter Include the following; lack of consensus surrounding 
Interviewer and Interviewee roles, variations in the presentation of self, and 
divergent interpretations of interactional rules. 

For all of these reasons, It ia believed that Interviewing could be 
most effectively employed when combined with other procedures so that the 
many problems associated with it may be mitigated. Among the best works on 
•interviewing are the following: Hyman, 1954; Richardson et. al., 1965; 
Garden, 1969; Denzln, 1970 and 1970a; and, Bogdan and Taylor, 1975. 

A third technique commonly associated with qualitative methodology 
although much less frequently employed than observation and interviews is the 
life history method. According to Denzin (1970a;415-416) the life history 
method involves the formulation of explanations: 
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that reflect one person's or one groups subjective experiences 
toward a predetermined set of events.. .the basic theme of the 
life history is the presentation of experience from the perspective 
of the focal subject or subjects. Their world most be penetrated 
and understood. Once it is entered, the observer lays out the 
critical-objective experience relevant to that world and then has 
his subject react to those events. In this way the subject's 
definition of the situation is compared to the objective events. 

Although the life history method is inherently an individualistic- 

j. 

psychologistic technique, it can be effectively modified to assume a more 
sociologically adequate form. This can be effectively done by recording 
a group of actors 1 perspectives on a single event to assess differences in 
interpretations of focal events as a function of social structural location. 
In this way a discarded methodological procedure may be revived and rendered 
sociologically useful. 

According to Becker (in Denzin, 1970a:419-428) , the life history method 
has a number of virtues. In the first place it is particularly useful in 
providing insights into the subjective side of institutional processes. In 
addition, because of its wealth of detail it can be useful in providing new 
insights for hypothesis formulation in areas of study having grown stagnant. 
Finally, life histories may provide useful information of social processes 
which other methods are incapable of revealing. Frequently charges of biased 
and unreliable data have been raised against users of life histories but 
Cavan, Hauser, and Stouffer (in Denzin, 1970a:436-441) note that reliably 
coded life history data can serve Important exploratory functions and can 
provide interpretations for quantitative findings expressed in statistical form. 

In addition to the works of Becker and Cavan et. al. cited above, see 
Blumer (1939), Allport (1942), and Gottechalk et. al.. (1945). More recent 
treatments of the life history method include Denzin (1970), and Schwartz and 
Jacobs (1979). 
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The three procedures briefly reviewed above are all reactive techniques. 
That is, the investigator becomes a part of the process he/she is investigating 
which may result in contaminating the phenomena under investigation. Clearly 
this direct intervention by the investigator in the process under study 
constitutes a major source of invalidity. To remedy this fundamental problem 
associated with reactive measures, qualitative researchers often employ a 
variety of nonreactive or unobtrusive measures. These measures include 
physical trace analysis, archival record analysis, and simple observations 
to include observations of exterior body and physical signs, expressive 
movement, physical location analysis, language analysis, and time sampling 
analysis. 

Physical trace analysis involves "...recording the incidence, frequency, 
and distribution of social acts toward certain social objects through time and 
in various situations" (Denzin, 1970:264). Ideally physical trace a^f lysis 
should be combined with other data on definitions and motivations of people 
underlying their actions toward social objects. 

Archival record analysis simply refers to the analysis of public and 
private archival records. Important here is the work by ethnooethodologist* 

r 

t such as Garfinkel, Cicourel and Kitsuse on the generation of organizational 

accounts which highlights the broader issue of the meanings which actors, 

differentially situated in organizations, impose upon such records. A 

succinct summary statement of Cicourel 1 s (1963 and 1968) contribution in this 

regard in offered by Denzin (1970:267): 

In analyzing such records, sociologists are dealing with the 
transformations of conversations, haphazard observations, and 
negotiated interpretations into formalized and consensual 
statements concerning the occurrence ~of a particular set of 
events, the motivations underlying such occurrences, and the 
actions of the record keener toward them. 

^> 
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Simple observations are a third type of unobtrusive measure refering to 
situations in which "...the observer has no control over the behavior or sign 
in question, and plays an unobserved, passive and unobtrusive role in the 
research situation" (Webb et. at., 1966:112). Among problems associated with 
simple observation measures is the fact that they are only appropriate for 
relatively small populations and public behavioral settings. Clearly these 
factors restrict their applicability. 

Finally, unobtrusive measures generally do not lend themselves to causal 
analysis owing in part to their extreme behavioristic bias. Such an important 
limitation can only be overcome by employing, together with unobtrusive 
measures, other procedures which are more suited to uncovering the subjective 
dimensions of social conduct. Fpr excellent discussions of unobtrusive 
measures see especially Oenzin (1970) axuTWebb et. al.(1966).. 



II. Qualitative Research in Knowledge Use and School Improvement 



A. The "Everyday" Character of Knowledge Production and Use 

If the extant literature on. knowledge utilization is any indication, the 
conclusion is inescapable that, notwithstanding the frequent employment of 
qualitative procedures generally for data collection purposes, utilization 
researchers have yet to come to terms with knowledge use as a form of 
practically rational conduct. A' noteable exception is the thought 
provoking- essay by Knorr (1981) entitled "Time and Context in Prac- 
tical Action: On the Preconditions of Knowledge Use." The 
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general lack of attention to the "practical problem" of use remains the case 
despite the fact that some researchers (Denzin, 1970; Holzner and Fisher, 1979; 
and, Larsen, 1979) have recognized the particular applicability of qualitative 
procedures for uncovering the subjective dimensions of use while study after 
study confirms the finding that knowledge is more apt to be used when the I 
user perceives a fit between that knowledge and various practical exigencies 
surrounding the performance of his/her organizational role. 

, This puzzling state of affairs signals the need for the development of 
models of the use phenomenon which might direct the researcher's attention to 
previously neglected subjective aspects' of knowledge use and to the employment 

of those particular methods especially suited to uncovering those aspects. To 
this end the model proposed herein Is offered. Simply put, the model seeks to 
uncover those elements constitutive of what we might refer to as the "everyday" 
character of knowledge production and use. Assumptions will be grouped under 
both- natural and theoretic stances taken toward the "everyday" character of 
knowledge production and use. From the standpoint of the natural stance, 
elements definitive of situated practices engaged in by actors involved in 
knowledge production and use will be identified and discussed. In addition to 
this, assumptions common to the theoretic stance w*ll be identified and 
discussed. Here we will be primarily concerned with conceptual and methodo- 
logical implications, derivable from the qualitative conception of knowledge 
use, of importance to the qualitative observer assuming the theoretic stance. 

According to Husserl (in Natanson, 1962) and Schutz (1962), the natural 
stance is that stance taken by actors in everyday life. "Taking the natural 
stance consists primarily in taking the standpoint of common sense, of acting 
within common sense" (Douglas, 1971:10). From this standpoint the everyday 
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world as experienced by the actor cakes on a taken-f or-granted character. With 
the assumption of the natural attitude serious questions regarding everyday 
experience are not raised (Douglas, 1970:14). In addition to the assumption 
that- the everyday world is taken for granted from the point of view of the 
natural stance, it is also assumed that the world as presented to ego is 
identical to the world as presented to alter. This assumption is what Schutz 
(1962) refered to as the "thesis of reciprocal perspectives" which constitutes 
a necessary precondition for the possibility of an intaraubjectively available 
world (see Garfinkel, 1967: chap. 2; and, Douglas, 1970:85n). 

For purposes here it is necessary to explicate identifiable properties 
of practically rational everyday activities and, more specifically, to treat 
knowledge producing and using activities as. particular exemplifications of such 
practically rational activities. Thus the question can be poied as follows: 
What are some identifiable features of practically rational activities in 
general and, of knowledge producing and using activities in particular? Answers 
to thl* question will serve as the assumption comprising the model of the 
"everyday" character of knowledge production apd use from the point of view of 
the natural stance and will be designated natural stance asstxnptions (NSA's). 

One of the most important distinguishing properties of practically 
rational conduct is intentionality . This property of common sense activity is 
highlighted by Schutz's (1962) dictum that our interests organize our constructs 
in the everyday world where component features of this world are addressed with 
"pragmatic motive" (Schutz, 1962:208-209; see also Heeren in Douglas, 1970:45) 
by actors mutually associated therein. The meanings attributed to objects 
attending everyday scenes of practically rational conduct are contingent upon 
the intentions, purposes or motives of actors engaged therein. This intentional 
character of practically rational conduct is of particular pertinence to the 
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study of knowledge use in a two-fold "sense. On one hand the production of 
knowledge in multiple forms as records, accounts, studies, reports etc. as 
well as the use of such forms of knowledge by persons other than the producers 
of them will always involve intentional action. On the other hand, decisions 
to use or not to use knowledge will always be, in part, based upon the criterion 
of practical adequacy. That is to say, from the standpoint of the natural 
attitude toward everyday life knowledge will be used to the extent that such 
use is practically adequate given the user's purposes at hand. 

Included among the many studies emphasizing the intentional character 

of the generation of records are the following: Cochran et. ai., 1980; 

Cicourel, 1968; Garfinkel, 1967: espec. chap. 6; MftCleary, 1975; and, 

Zimmerman in Douglas, 1970:221-238. As Cochran et. al. (1980:6) suggest, 

records are proactive insofar as "...a record keeper's plans, goals, intentions 

and assumptions precede and therefore shape the record. . .People assemble and 

use records with some goal In mind." Among the many purposes for which records 

are generated \ % Cochran et. al. (1980: 13) enumerate the following: 

...to maintain interpersonal relationships, control the behavior 

of others, protect oneself, save time, eliminate busy work, 

avoid unwanted scrutiny, exercise direction, over one's work, 

document cases that can be successfully resolved, and document 

that work has, in fact, been done. j 

This finding is consistent with analyses of-rrecords produced in juvenile 

probation agencies (Cicourel, 1968; and, McCle^ry, 1975), public assistance 

agencies (Zimmerman in Douglas, 1970:221-238), and in mental health clinics 

(Garfinkel, 1967i^chap. 6). Among the'six "normal, natural problems" of 

clinic records which Garfinkel (1967:192ff) identifies, two are of particular 

significance to the problem of the intentional character of record keeping. In 

the first place it; is imperative to note that some self -reporting forms 

constitute rules of reporting conduct as a-w>rk obligation. Thus it is 
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reasonable to assume that actors' accounts reported in^such forms will reflect 
the purposeful attempt to verify compliance with work oMlg^^fcms. This 
suggests a second and related problem of records identified by Garfinkel. 
That is the fact that records entail risks to per^nal and organizational 
careers. Hence one would suspect that recprd producers would seek to avoid 
such risks to themselves and furtHer that the information contained in" the 
records would reflect this intention. . - 

That the production of knowledgeable accounts is an intentional process 
is a face suggested by the model of the "everyday" character of knowledge 
production and use and a fact which utilization researchers would do well to 
remember. Whether the production of records is taken as the object of inquiry 
or the records produced are simply treated as data in utilization studies, the 
researcher is well advised to attend to their proactive character. The 
intentional character of the produced records constitutes a threat to validity 
when it. is ignored. 

And if knowledge production is intentional or purposive* it is no less 
true of knowledge use. Decisions to use or not ^o use knowledge will 
ultimately rest upon the intentions of the actors as they attempt to solve the 
practical problems of their day-to-day routines. Failure to account for these 
practical problems confronting potential users by change agents and researchers 

"it" 

will result in the imposition of a variety of resistances to utilization 
(Hofler, 1981). 

I. '• 

The above discussion then yields our first natural stance assumption: 

NSA : THE PRODUCTION AND USE OF KNOWLEDGEABLE ACCOUNTS, 
v 1 AS A FORM OF PRACTICALLY RATIONAL CONDUCT, IS AN 

* INTENTIONAL PROCESS. ACCOUNTS PRODUCED REFLECT 

THIS INTENT I ONALITY . 

In addition to intentionality, another property of practically rational 
conduct is indexicality which refers to the broader contextual determination of 
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meaning of constituent elements of everyday interaction scenes. This property 
expressed in Schutz's (1962) notion of the .paramount reality of the Lebenswelt 
is reflected in two principles common to phenomenological sociology. The first 
of these is the principle of the integrity of the situation which holds that, 
"concrete human events are always to some degree dependent upon the situational 
context in which they occur and qan adequately be explained only be taking 
into consideration that situational context" (Douglas, 1970:37). Related to this 
general principle is the more specific principle of the contextual determination 
of meaning .which holds that, "the context within which a given statement or 
action occurs is of fundamental importance in determining the meanings 
imputed to it by the members of society" (Douglas fc 1970,37) . 

There are two types of contexts which determine meanings: the linguistic 
context and the practical use context. Meanings of items are linguistic 
context dependent insofar as they are determined by other linguistic items, 
terms, statements, occurring in' the same communicative situation. This 
indexical quality of language was first analyzed by Bar-Hillel (1954) and 
subsequently taken up in the work of linguistic ethnomethodologists such as 
Garfinkel, Cicourel, and Sacks. Meanings are practical use context dependent 
when they are determined by nonlinguistic aspects of the situation in which 
they occur. Such nonlinguistic situational aspects include time and place 
together with tacit assumptions about actors or objects of the interaction 

situation'. 1 

i 

With respect to knowledge use, the indexical character of practically 
rational activities suggests that the meanings attributed by actors to 
features of knowledge use processes are determined by the broader contexts 
within which such concrete knowledge using activities occur. These broader 
contexts include a wide diversity of elements which combine to structure the 
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more concrete knowledge using practices. These elements Include time, place, 
personal and organisational commitments, status-roles of participant actors 
and most importantly fetors' frames of reference which embody assumptional 
structures pertinent to the actors' use of 'knowledge. These assUptional 
structures include truth "and utility tests which actors employ in their 
knowledge appraisals (Weiss and Bucuvalas, 1980; and, Holzner and Fisher, 1979) 

The indexical nature of practically rational conduct signals the 
overall interpretive character of knowledge use when use activities are 
regarded as forms of practically rational conduct. Knowledge use will always 

involve^Tsojnpeting interpretations held by a variety of stakeholders as to the 

t • 

"meaningful" content of ' knowledge products (Dunn and Holzner, 1980:3-4). 
One of the central tasks confronting utilization researchers is the unearthing 
of such interpretive structures or, we might say, attending to the indexical 
character of practically rational knowledge use activities. 

Thus considered, our second natural stance assumption might read as 
follows: > 

t 

NSA : KNOWLEDGE PRODUCTION AND USE, AS A FORM OF 
2 PRACTICALLi RATIONAL CONDUCT, IS, INDEXICAL 
IN CHARACTER.- IT INVOLVES THE IMPUTATION OF 
MEANINGS TO KNOWLEDGE PRODUCTS AND OTHER OBJECTS 
OF CONCRETE INTERACTION SCENES BY ACTORS * 
• ASSOCIATED THEREIN. 

This second assumption is integrally related to a third assumption 

concerning the interactional quality of practically rational conduct. The 

world as presented to actors from within the standpoint of everyday life is 

experienced interaubjectively. While the indexical property of practically 

rational activities in general and, more specifically, of knowledge producing s 

and using activities so considered highlights the importance of the broader 

contextual determination of meanings of such activities and their constituent 

components, the interactional quality directs our attention to the negotiated 

character of these meanings and interpretations imputed to such activities 
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and constituent components thereof. \ 

Knorr (1981:17) provides the best ^statement on the interactive nature 
of knowledge use activities to be found in the utilization literature 1 . She 
nor^s: 

...the basic observables of practical action will have to be * 
conceptually specified as interacts rather than acta. This 
entails that decisions. . .oust be seen as in principle negotiated, 
which means that they arise out of the consideration of or out 
of direct confrontation with other relevant agents. And issues 
like the practical relevance or political feasibility of research 
results must be seen not only as interpreted in context, but as 
resulting from a direct or indirect confrontation of opinions in 
a process which is fundamentally a process of contestation and 
negotiation. 

Appreciation of, the interactive character of knowledge use activities 
then directs our attention to both the contested nature of claims employed 
in knowledge appraisals and the dynamic nature of frames of reference within 
which such claims «re embedded.' On the one hand the interaction focus is on 
the interactants themselves or the Variety of stakeholders mutually associated 
in interactive processes of knowledge transactions. Here it is Important to 
note that insofar as knowledge use entails changes in familiar behavioral 
patterns of potential users, such changes, anticipated or real, may fostsr 
resistance to use and provide a basis for contestation. Hofler (1981) 
provides useful insights here. On the other hand the interaction focus is on 
the knowledge claims themselves- and the question becomes one of how claims 
employed in knowledge appraisals are interrelated and how knowledge use varies 
as a function of different patterns of interrelationships among claims. 
(This is the multiattribute problem specified in'Dunn and Holzner, 1980). 
Together then these issues suggest the Importance of the interactive 



9 

ERIC 



1X7 



21 



property ojf- pre«4d.cally- rational knowledge use activities, and provide as 
with the third natural stance assumption of the "everyday" character of 
knowledge production and use model: * 

NSA : KNOWLEDGE PRODUCT-ION AND USE, AS PRACTICALLY RATIONAL 
3 ACTIVITIES, ARE INTERATIVE. DECISIONS REGARDING USE 
TOGETHER WITH OTHER MEANINGS IMPUTED TO ELEMENTS 
CONSTITUTING PRODUCTION AND USE ACTIVITIES ARE, IN 
PRINCIPLE, NEGOTIATED) BY ASSOCIATED INTERACTANTS . 

These three assumptions concerning knowledge production and use activities 
are grounded in the natural stance position taken toward sjjch activities 
regarded as practically rational conduct. Specifically, these assumptions 
taken together paint a picture of selected identifiable properties of the" 
knowledge production and use phenomenon considered as a phenomenon of 
everyday life. 

Such consideration of knowledge production and use activities as a 
phenomenon of everyday life involves a related set of assumptions concerning 
both conceptual and methodological Implications following from this particular 
view of the phenomenon. Here the emphasis is upon the treatment of everyday 
life and, more specifically, of knowledge production and use activities 
considered as such as a phenomenon or object of inquiry. The observer stands 
apart from the everyday world and investigates features thereof from the 
theoretic stance. In the words of Douglaa (1971:10), "...taking the theoretic 
stance consists in standing back from common sense and studying common 
sense to determine its nature." Thus specification of additional assumptions 
comprising our model of the "everyday" character of knowledge production and use- 
involves taking the observer's point of view toward the phenomenon of "everyday" 
knowledge production and use activities. That is, we will identify and 
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discuss selected conceptual end methodological implications which observers 
or utilisation researchers should attend to, given the"everyday" character 
of knowledge production and use. These assumptions will be designated 
theoretic stance assumptions (TSA's). 

Treating knowledge production and use as practically rational activities 
or as an "everyday" phenomenon signals the need to develop particular con- 
ceptual formulations appropriate thereto. It is worthwhile noting that the 
utilisation literature has generally been characterized by a persistent 
neglect of the "practical problem" of knowledge production and use, a 
neglect reflected in a variety of elements of the conceptual apparati to be 
found in this literature. The issue of the ^two comnunities" metaphor 
(see Caplan, 1979; Caplan et. al., 1975; Dunn, 1980; and, Rich, 1979) may 
serve as an interesting case in point. 

The two communities metaphor addresses attention to the relationship 
between the social scientific community of knowledge producers and the 
policy making conmunity of knowledge users. The metaphor has the benefit of 
highlighting significant subjective features of knowledge use by treating 
both communities as ensembles of different meaningful claims employed in 
knowledge appraisals by actors differentially located in social organizations 
(see Dunn, 1980). Yet despite this significant contribution provided by 
this conceptual metaphor, a contribution which should be duly emphasized, use 
of the metaphor entails certain problems given the consideration of knowledge 
production and use as an "everyday" phenomenon. Use of the metaphor does not 
go far enough in addressing the everyday character of knowledge producing and 
using activities. 
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Specifically, use of the metaphor involves the unfortunate tendency to 
radically separate knowledge producing and using activities. On one hand 
producers are regarded as being located in particular organizations where they 
employ a, specific set of claims regarding knowledge while on the other hand 
users occupy positions in different organizations and employ different 
knowledge claims. So considered, knowledge transactions assume the form of 
transfers of knowledge products or bits from one location where they ere 
produced by one set of actors to another location where they are used by 
another set of actors. 

Such a conception is fraught with a variety of shortcomings, not the 
least importsnt of which is the denial of the benefits associated with reflexive 
sociological inquiry. When the rt everyday" character of knowledge producing 
and using activities is taken seriously, the vast gap between production and 
use suggested by the two communities metaphor disappears. Regarded as an 
everyday phenomenon, knowledge production is at one and the same time 
knowledge use. Afterall, the production of knowledge will always have for 
its end certain purposes or alms arising out of the concrete occasions of that 
production. Even among our producers of social scientific knowledge, the 
knowledge produced will simultaneously be used to fulfill a variety of 
purposes such as procurement of research monies, education and professiona- 
lization of students, career advancement and professional certification Just 
to name a few. And decisions with reipect to whst knowledge in produced-used 
will be in pert s function of the social scientist's determinetion of his/her 
own purposes st hand. 
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Thus the "everyday" notion of knowledge production end uae invites the 
utilizetion researcher to reflexively consider his/her own conduct. Such 
'-consideretion may be .sen to fulfill « two-fold purpose. On one hand it 
should s.rve to sensitize the researcher to importent features of the user's 
conduct while on the other hand alerting him/her to previously unrecognized 
cherecteristics of his/her own conduct. From this point of view the research- 
er's own precticelly retionel ectivity becomes e topic of investigetion 
es well es e resource for investigetion, «s he/she renders observeble the 
reflexive cherecter of prectical ectivities. Awereness of this reflexive 
property of precticel ection is e fundamentel precondition of jthnomethodo- 
logicel inquiry es exemplified in the works of Gerfinksl and Cicourel. 

Attention to such reflexivity is of even breeder significence in thet it 
pointa to the need for the development of grounded theory end concepts 
(Gleeer end Streuss, 1976; end, Dunn end Swi.rczek, 1977) %hich have their 
underpinnings in precticel action rether then ungrounded conceptuelizetions 
which obscure the "everydey" character of knowledge production and use. 

"Ha 

The ebove discussion suggests two theoretic stence essuaptlons: 

TSA : CONCEPTUALIZATIONS REGARDING KNOWLEDGE USE WHICH , 
1 ARE GROUNDED IN PRACTICAL USE ACTIVITIES, THEREBY 
MORE ADEQUATELY REFLECTING FEATURES OF SUCH 
ACTIVITIES, SHOULD BE EMPLOYED OVER CONCEPTUALIZA- 
TIONS WHICH DISTORT THE COMMON SENSE CHARACTER OF 
USE ACTIVITIES. 



TSA 



PRACTICAL ACTIVITIES OF UTILIZATION RESEARCHERS 
SHOULD BE REGARDED AS TOPICS OF AS WELL AS 
RESOURCES FOR INVESTIGATION. 
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In addition, treating knowledge production end use as practically 
rational activities suggests the need to employ specific methodologies 
appropriate to the "everyday" character of the phenomenon. On one hand, 
insofar as knowledge production and use activities are subjective in 
character involving actors 1 intentions, purposes, aims, etc. methodological 
techniques which have as their aim the uncovering of the subjective 
interpretations of action will have to be employed (Larsen, 1979; and, Dunn, 
1980). It should be emphasized that such qualitative data are pirt^l arly 
vulnerable to assorted contaminating influences of the researcher and the 
investigative process itself. And taking into account this special character 
of qualitative data, only those methodological procedures which retain the 
integrity of the phenomena should be employed. 

In addition to being subjective, problems of knowledge use tend to be 
interdependent, thus methodological techniques should be holistic in 
character. They should take account of the interdependencies among knowledge 
use problems (Dunn, 1980). This brief discussion yields our third theoretical 
stance assumption: * 7 

TSA : METHODOLOGICAL TECHNIQUES EMPLOYED IN UTILIZATION 
3 STUDIES SHOULD BE SUBJECTIVE, HOLISTIC, AND 
SHOULD RETAIN THE INTEGRITY OP THE PHENOMENA. 

Together then these assumptions comprise our model of the "everyday" 

character of knowledge production and use. They should not be regarded as 

exhaustive but merely as preliminary attempts to outline a particular view 

of the phenomenon of knowledgk^iee whi,ch has generally been absent in the 

utilization literature. Thus the model invites the researcher^ attention to 

the "everyday" practically rational character of the knowledge use phenomenon. 
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Having/tpecified the internal components of the model it Is necessary to 
proceed to the assessment of the extent to which the application of qualitative 
methodological procedures in studies of knowledge use and school improvement " 
is consistent with elements of the model. Our review of this research 
literature will utilize the three knowledge use research problems discussed 
in the above as an organizing frame. 

B. Review of Research on Knowledge Use and School Improvement by Research 
Problem with Attention to Qualitative Procedures 

The vast body of educational research literature bears witness to a 
pronounced interest in and employment of a variety of qualitative methodologies 
(see Chart I prepared by Wolcott (1980) on following page) . One of the 
methodologies most frequently used by educational researchers is ethnography 
which is now more commonly found in the modern classroom than in the preliterate 
tribe (sec Erikson, 1973; Everhart, 1975, 1976; Lutz and Ramsey. 1974; 
Mulhauser. 1975; Rist. 1975; Stake, 1978; Wilson, 1977; and, Wolcott, 1975). " 
And while the clessroom has served as the site for the employment of more 
conventional ethnographic methods by educational researchers, it has elso been 
the object of study by growing numbers of ethnomethodologists wielding a 
variety of unconventional research tools. Whether the topic is differences in 
reality construction between children and edults (Cicourel et. el., 1974; 
Handel, 1972; Jennings, 1972; Leiter, 1971; MacKay, 1973, 1974; mehan, 1973; 
and, Roth, 1972) or the social construction of student performences 
(Cicourel and Kitsuse, 1963; and, Leiter in Cicourel et. al., 1974), the 
classroom has become a common site of ethnomethodological investigation. 

It could be expected that such marked attention to the uSe of qualitative 
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procedure* in the educational research literature generally would be paral- 
leled by employment of qualitative procedures in the more specific area of 
knowledge utilization research in «**c*tional settings. While this is 
precisely the case *Twe shall Immediately see below, those qualitative 
procedures employetT in utilization studies are of the conventional variety. 
Indeed, more uncotanon ethnomethodological techniques are conspicuous bjfr tfielr 
absence in this literature. 

1. The Criterion Problem: Research Review 

The nature of the criterion problem is best capty*d in the question 
posed by Larsen (1980), "Knowledge^ 1 11 izat ion: What im i,t7 M . li|jhile at 

first glance this question seems rather simple, upon (further reflection it 

\ \ 

becomes apparent that this question implies a host of "other questions; the 

1 N \ \ 

criterion problem is a composite including a numbe* of mori\specif ic 

problems. And the application of qualitative procedures would* appekr 
to be particularly appropriate in utilization researchers' attempts \co 
resolve these problems, tt is necessary to turn now to a review of 
of these representative attempts. Corbett and Firestone (1980) identify 
four strategies for increasing the use of research in policy making. 
These include the enlightenment-transfer, engineering- transfer, enligh-\ 
tenment- transaction, and the engineering- trans act ion strategies. The 
enlightenment-engineering designations characterize differing research 
designs employed, basic and applied respectively, while the transfer- 
transaction designations describe differing -methods of disseminating in- 
formation. The authors advise qualitative researchers especially tcQ 
use either of the two transfer strategies. 
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An Interesting example of the effective employment of qualitative 
procedures in an analysis of the knowledge transfer process in educational 
settings in provided by William Conner (1980) ^ a colleague of Corbett and 
Firestone at Research for Better Schools, Inc. (RBS) • Donner (1980)' 
reports on a two year longitudinal study of five schools conducted by RBS. 
The effort involved the development of research-based approaches to 

facilitate the Improvement of the schools' programs, in two areas, basic 

9 1 . 

skills and career education. Three research questions were posed: 
I) What is the nature of the knowledge transfer process? 2) How is 
research knowledge combined with "practical" knowledge in the transfer 
ptocess? 3) How is research knowledge interpreted, understood and used by 
teacher^? 

Although the third research problem is related to the criterion 

... ' c 

problem, it deals more specifically with the diversity problem. Hence our 
treatment of this Aspect of Donner 's work will be taken up in the following 
section. It is interesting to note that the specification of these 
particular problems for research reflects a prior appreqiatibn of the 
"everyday" character of knowledge use. And posing the questions In these 
terms Implies the particular applicability of qualitative procedures. 

The two data gathering procedures employed were observation and 
interviews. In the first year field workers observed project meetings 
with RBS representatives an4 participants from participating school 
districts. In the second year researchers spent more time on-site in 
the participating schools. Throughout^feoth years teachers, administrators 
and linking agents were interviewed both formally and informally. Each 
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field worker concentrated on one or two schools- and field notes were made 
available to the entire research staff who conducted ongoing discussions 
_ about their findings. 

Researchers found the knowledge transfer process to consist of four 
analytically distinct stages that overlapped in time: inhouse development 
of component approaches and materials, presentation by linkers, trial and 
use by- teachers, and feedback to developers for revision. Furthermore, 
researchers found that practical knowledge entered into every stage. As 
Donner (1980:54) concludes: 

' \ ' ' 

The entire process of research into practice can be viewed as a 
tension between two poles. At one pole there is research and 
knowledge about a content area. At the other pole there is the 
^ practical circumstances of schools and classrooms. For new 

knowledge to be used, it must fit into the practical circum- 
stances of schools and classrooms. 

Although Donner is not more explicit about the specifics of his 
data collection techniques, it is clear that the "practical" character of 
the knowledge transfer process could only be uncovered by the use of * 
qualitative procedures. Thus his work at least serves to demonstrate a 
fundamental principle underlying qualitative methodology; that is, 
methodological techniques are problem-specific. 

An interesting application of the focused interview may be found in 
the work of Hall, Loucks and their associates at the Research and Development 
Center for Teacher Education at the University of Texas at Austin. Research 
at the Procedures for Adopting Educational Innovations Project of the R&D 
Center focuses on a model of innovation adoption, the Concerns-based 
Adoption Model (CBAM) t 1 The model is composed of two dimensions: Stages of 
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Concern about the<*4ne0vation (SoC) and Levels of Use of the innovation 
(Loll). Here we will be primarily concerned wit* the LoU dimension. 

Hall, Loucks and their associates conceive of innovation use as an 
eight stage process which ranges from lack of awareness of the innovation 
to active searching for alternatives to or modifications of the innovation. 
As Hall et. al. (1975) suggest: 

Levels of Use are distinct states that represent observably 
different types of behavior and patterns of innovation use 
as exhibited by individuals and groups* These levels charac- r ) 
terize a user's development in acquiring new skills and 
> varying use of the innovation. Each level encompasses a range 
of behaviors, but is limited by a set of identifiable 
decision points (see Chart II - LoU Chart on following page) . 

To measure individual user's LoU the researchers employ the focused 

interview technique characterized by specific objectives for questioning 

- x 

though devoid of standardized questions. More rigorous standardization 
of questions typica^ of the structured interview is eschewed in favor of 
greater flexibility given the complexity of the LoU concept. This techr 
nique permits 1 more true-to-life responses and greater individualization of 
follow-up questions to respondents, a necessary precondition for fulfillment 
of Interview objectives. 

The LoU interview is also characterized by its generic quality. 
That is to say that it is not specific to any single iiraovation but rather 
the questions can be used for different innovations. Another interesting 
feature of the interview is its branching format. The interviewer asks * 
questions to locate the respondent at specific decision points and then 
intensively probes .the model's categories to more precisely locate the 

i 

individual's LoU, (see Chart III on p. 42). 
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After the interview data is collected it is rattd and rater relia- 
bility is assessed in two ways. "Individual raters are evaluated through 
examination of their percent of agreement with other raters. The syst«n 
as a whole is evaluated through recourse to standard reliability coef- 
ficients" (Loucks et. al., 1975:46). Hall and Loucke (1977) also report 
on a validity study of the LoU interview procedure which employed ethno- 
graphic observation. As Hall and Loucks (1977:267-268) report: 

Based on the LoU interview ratings, seventeen teachers 
representing a stratified sample including all LoU levels 
were selected for ethnographic observation. Ethnographers 
spent one full day with each teacher. . .the ethnographers 
assigned an LoU rating to the teachers and developed a set 
of written protocols. 

Two major comparisons of the data were made as estimates 
of the validity of the LoU interview: (1) the ethnographer's X 
LoU rating of the teacher was compared with the consensus ) 
LoU interview rating, and (2) the consensus ratings of / 
Independent readers of the protocols were compared with the / 
consensus LoU interview rating. f / 

The correlation coefficient for the first comparison was .98, while for 
the second it was .65. 

The work of Hall^and his associates then demonstrates the uniquely 
enlightened use of conventional qualitative techniques both for primary 
data collection purposes and for validity assessment of research 
instruments. Such work constitutes a major contribution to the field of 
knowledge use and school lmpjrovment . 

i 

This work is rather unique in another way as well. The emphasis 
is clearly upon an overarching conception of use which treats different 
use behaviors as discrete phases in or dimensions of use in general. 
The Implication is that users always and everywhere, regardless of their 
characteristics or characteristics of the innovation/knowledge itself, 
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will progress In due course through the stages of the use process, the 
central focus is on use in general rather than on the user in particular. 

This snphasis is rather exceptional in the knowledge use and school 
improvement literature where attention to multiple types of different and 
specific uses of knowledge seen to vary as a function of the user's 
social structural position is more commonly found. A number of studies 
identify different use patterns with different user attributes. Among 
these are included the following: Berman and McLauglin, 1977; Culberteon, 
1977; Daft and Becker, 1978; Donner, 1980; Emrick and Peterson, 1978; 
Froh and Muraki, 1980; Fullan, 1980; Hood et. al., 1976; Louis, 1978; 
Miles et. al. , 1978; Rosenblum and Louis, 1979; and, Sieber, 1981. 

While Emrick and Peterson (1978), Miles et. al. (1978), Louis (1978), 
and Rosenblum and Louis (1979) emphasise the role of leaders, especially 
school administrators, in supporting the utilisation process, Berman and 
McLauglin (1977), Daft and Becker (1978), Hood et. al. (1976) focus on the 
teacher as knowledge user. A good review of the literature on teacheSfC 
principals, and superintendents as knowledge use agents is provided in 
Fullan (1980). 

All of these studies smploy conventional survey and/or observation 
and interviewing techniques for data collection purposes in their research 
designs. Responses to questions about different types of use are seen to* 
covary with specific attributes of the user to include status-position, 
educational level, prior experience with utilization efforts, etc. It 
should be noted that the way in which positional location of the user is 
Refined has important Implications for the level of precision in the 
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specification of us* behavior. Rather than focus on mars incumbency 
in a status-position, it is necessary to regard the actor's self-definition 
of his/her role as veil as the multiple practical problems associated with 
the performance of particular roles as Implying important use-pertinent 
Implications. These variables could be expected to influence use behavior 
by either constraining or facilitating use as veil as by serving as 
Inducements to the modification of knovledge-in-use in a variety of ways. 
For example, studies of teachers (Jackson, 1968 and Lortie, 1975) emphasize 
the pragmatic, practical, and concrete character of "teacher culture." 
And studies of teachers as knowledge users document the constraining 
influence vhich practical considerations of the teacher role have on use 
behavior (see especially Donner, 1980; Froh and Murakl, 1980; and, Fullan, 
1980). 

This suggests the need to develop broader conceptions of user 
attributes generally and of social structural location specifically. 

I 

Particularly, utilization researchers must examine the subjective tlentntf 
in use behavior, elements vhich highlight the interpretive character of 
knowledge use. This is precisely the focus of our second problem, the 
diversity problem. 

2. The Diversity Problem: Research Review' 

The diversity problem emphasizes the interpretive character of 
knowledge use by focusing upon the assumptions, embedded in frames of 
reference, which actors employ in their appraisal* of knowledge. Assumptions 
may be regarded as taking the form of a variety of tests, relevance, truth, 
and utility, employed in knowledge appraisals (see Dunn, 1981; Dunn and 
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Holzner, 1980; Holzner and Fisher, 1979; and, Weiss and Bucuvalaa, 1980). . 



The diversity problem, or the assessment of these assumptions, entails a 
number of methodological implications for utilization researchers. On one 
hand it is necessary to emplby techniques to determine the extent to which 
such assumptions are contingent upon attributes of the knowledge or 
information transferred itself, as well as attributes of the broader 
contextual situation of use. Here attention must be paid to the broader 
aspects of social structural location to include purposes, intentions, 
and practical considerations adhering to actors 9 status-roles. Qualitative 
procedures are particularly applicable here. On the other hand, novel 
directed techniques which have as their explicit purpose the uncovering 
of actors* assumptions about knowledge use will have to be borrowed from 
related fields or developed de novo by utilization researchers. 



(observation and interviews) to the problem of the practical contextual 
basis of knowledge use-pertinent assumptions may be found in the work of 
Donner (1980) mentioned above. Attention to the diversity problem is 
evinced in the formulation of one of his research questions viz. "How 
is research knowledge interpreted, understood and used by teachers?" 
(Donner, 1980:4). Donner notes that a number of practical problems are 
inhibitive of more extensive knowledge use. Among these are included the 
^following: amount of time required to Implement project tasks; complexity 
of project tasks and procedures; prior negative experiences with use 
efforts; conflicting definitions of project elements between researchers 
and users; and, collegial pressure among potential users. 




application of conventional qualitative procedures 
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Interpretation* of knowledge then will always be embedded in the 

V 

myriad of practical considerations surrounding the performance of particular 
roles. As mentioned above, a number of studies (Donner, 1980; Froh and 
Muraki, 1980; Fullan, 1980; and, Sieber, 1968) focus upon-t** practical 
problems surrounding the teacher 1 a role and the consequences of these on 
knowledge utilisation. This suggests the need to broaden the definition 
of status-role to include sets of role-specific practical problems 
adhering to specific roles which serve as foundations upon which arise 
more specific teats pertinent to assessments of knowledge claims. 
Qualitative field methods are uniquely suited to uncover these practical 
elements of use, 

An interesting directed technique which may be of use in uncovering 
assumptions pertinent to knowledge use may be found In the related 

if 

evaluation research literature, specifically in Policy ^plications / 
Analysis (PIA) as proposed by Madey and Stenner (1980). As the author* 
indicate, "PIA is based on the assumption that a more responsive and 
useful evaluation will result by understanding both the policy cqntipct 
within which the evaluation is commissioned and also the questions being 
posed by actors within that context" (1980:3). "PIA is a six-itep 
process designed to explicate the information requirements of kiy information 
users..." (Madey and Stenner, 1980:5)., It builds upon two futures 
methodologies, the delphi technique and scenario writing. We wiluL para- 
phrase the six-step process below, changing terms to make the description 
more applicable to knowledge utilization* while dropping the sixth \step as 
it is inapplicable. * \ 

I , \ 
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Step 1 : Statements of hypothetical outcomes which may result 
from the use of the knowledge product are generated. 

Step 2: Using these outcomes a written exercise is prepared 
for later use with a panel of stakeholders. The 
/ e*ercii%*J^comprised on the following: 

- hypothetical outcomes or "scenarios" with guidelines 
for responding to them. V 

- opportunity for stakeholders to generate their own 
favored outcomes with modification of and non-use 
of knowledge as open options. 

"* *n evaluation of the exercise. v. 

Step 3: A carefully selected panel -Of stakeholders Is identified. 

Step 4: Each panel member completes the exercise responding ta 
the hypothetical outcomes in terms of the following: 

- how likely he/she faels that the outcome will 
result from utilisation of the knowledge. 

- how he/she feels the outcome would Impact upon 
him/her self or others. 

Step 5: Responses of individuals are analyzed to: 

- clarify the expectations of relevant stakeholder 
groups. > 

- delineate the context within which use of the 
knowledge product is embedded. 



This process would appear to be a useful technique to surface user 
assumptions and it could be further adapted for such purposes. Yet it is 
not without its problems including its highly obtrusive character, the 
time required to carry it out, and the many practical problems surrounding 
panel selection. Yet despite these problems, such directed techniquaa- 
should not be overlooked by utilization researchers in attempting to deal 
with the diversity problem. 
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3. The Multiat tribute Problem! Research Review 

While the diversity problem is concerned with the identification and 
measurement of individual standards for assessing knowledge individually 
considered, the multiattribute problem focuses on the interaction effects 
between/among different knowledge appraisal standards. The main point here 
' is howNcnowlsdge appraisal standards interact with one another in decision 
making processes regarding the use of knowledge. 

Realization of the multiattribute problem is conspicuously absent 
in the knowledge use and school improvement literature and while such a 

situation prohibits assessment pf qualitative procedures applicable to this 

\ r 

problem, it does not rule out our ability to propose suggestions foi; both 
conceptualization of the problem and appropriate methodological procedures 
pertinent thereto. - 

Again we turn to the notable work of Hall and his associates. 
Interestingly, their Stages of Concern about the innovation (SoC) component 
of the CHAM model has Implications for conceptualization of the multi- 
attribute problem. According to Hall et. al. (1979), innovation adopters 
(read: Knowledge users) go through a series of stage of concern (seven in a£ 
about the innovation/knowledge. Concerns Vary across a linear continiium; 
at first adopters/users are concerned about the impact of the innovation/ 
knowledge on themselves and later they progress to a stage of concern for 
the Impact of the innovation/knowledge on significant others in their 
environment. If we treat Hall's notion of concerns, which is defined 
broadly in any case, as inclusive of assumptions regarding uto, truth, 
utility, relevance tests, this conceptualization suggests that these 
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components likewise may be seen to interact In a developmental process. 

A major problem with this conception is that It treats assumptions 
as attributes of an individual user - knowledge product analytical unit. 
And of course assumptions are shared entities. Thus attention to the 
shared character of assumptions and use processes ih general should be 
reflected in conceptions employed by utilization researchers. If the 
Hall SoC concept had^ a collective dimension it might be more fruitfully 
applied to the mult iat tribute problem. Yet despite this shortcoming, 
the SoC Concept may provide some useful insights for conceptualizing this 
problem (see Hall et. al., 1979). C 

C. Qualitative vs Quantitative Methodology: Toward a Rapprochement 

t The brief review of qualitative' procedures emplbyed in research on 
knowledge use and school improvement offered above suggests a number of 
conclusions. In the first place it is clear that mere use of conventional 
qualitative procedures is, not synonymous with appreciation of the practically 
rational character of knowledge use activity. While many studies in the field 
employ an array of conventional qualitative techniques, few treat seriously 
the "everyday" character of knowledge production and use. In addition, the 
above review attests to the difficulty of systematically analyzing the use of 
qualitative methods. Such methods do not easily lend themselves to clear 
formalization or codification. This is not so much an artifact of the 
methods themselves, I think, as it is a matter of professional writing style, y 
Qualitative researchers rarely elaborate* systematically on the specifics of 
"their research methods. Rather, almost apologetically,' they identify them 
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and proceed with the research findings., We desperately need*a science of 
the systematica of method including qualitative methods to compliment the . 
study of the systematica of theory that Merton has called for. 

That this is so is doubtlessly the result of the particular scientific 

culture within which we find ourselves, roue whidh prescribes the use of 

.$ 

quantitative methods while deriding the use of qualitative procedures. Yet 
the battle over qualitative or quantitative methods obscures the reciprocal 
contributions which enlightened employment of both may provide. What is 
required in utilization studies and elsewhere is a strategy of multiple 
triangulation as proposed by Denzin (197Q) (see also Jick, 1979). The 
combination of multiple observers, theories, methods, and data sources 
provides increased opportunities for 1 the validation of social scientific 
findings. In addition to the significant accomplishment of cross-validation 
(see here Campbell, 1974) provided by triangulation, the cross-fertilization 
of qualitative and quantitative procedures results in a number of benefits 
at each stage of the research process (see Sieber, 1973; and^, Madey, 1980; 
and Chart IV on following page) . 

Utilization researchers would do well to rise above the hackneyed 
hue and cry of the quantitative vs qualitative fracas and reap the valuable 
benefits of triangulation. Such a strategy would result in a noticeable 
Improvement in the quality of utilization research and, one hopes ultimately, 
in. the Improvement of professional practice In the educational community 
and -elsewhere. " > 
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Co ntributions of S^llMtlv* v«thoda t» - ,..^ n|1| 

Design 

Qualitative aethod. can enrich quantitatlv. design, by taproving, 

(1) the saapllng fraoework 

(2) the overall study design 
, Data Collection 

Qualitative rethod. can enrich Quantitative d . t . collection by improving: 

(1) th* % instrument package 

(2) instruoentation aciainisuation ' 

Analysis . ^ 

Qualitative tcthod. can enrich quantitatlv. analysts by providing 

(1) a conceptual framework to guide the analysis 

(2) verification of quantitative findings 

(3) index construction of quantitative analyses 

(4) external validation 

^ (5) case study Illustration 

(6)^ clarification of quantitative findings 

Contributions of Oi^Mrirf,, Methods to Qualitative 

Pesljr n 

Quantitative nechod. can enrich qualitative design, by identifying: 

(1) . representative cases, to serve the goal of generali 2 afrilit 7 

(2) unrepresentative cases, to refine nod.ls and theories 
Data Collection 

Quantitatlv nethod. can 'enrich qualitative data collection by providing: 
(1) leads for later interviews and observations 
(?) information about overlooked subjects 
(3) correction of the "elite" bias 
Analysts 

Quantitative method, can contribute to the understanding of qualitative 
analyses by: 

Cl) correcting the holistic fallacy 

(2) verifying qualitative interpretations 

(3) casting new light on field observations 



Exhibit 1. Su=*ary of Benefits Obtained by Integrating 

Methods in Design, Data Collection, and Analysis 

from MADEY, D. (1980) "Some benefits of integrating qualitative and quantitative 
methods in program evaluation, with il lustrations." Educational Evaluation 
and Policy Analysis (J orthc<4ing) • 
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ABSTRACT 



The purpose of this paper Is to explore 
potential applications of social network 
analysis^ to research on knowledge use and 
school Improvement. The paper (1) outlines 
conceptual and methodological bases of 
social network analysis; (2) reviews, some 
relevant past applications of social net- 
work analysis tb the study of various 
facets of social structure and behavior; 
and (3) proposes and justifies the Idea 
of "epistemic network analysis" as an 
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INTRODUCTION 

0 

In this paper we explore potential applications of social network analysis 
to the study of knowledge use and school improvement. Our attention to social 
network analysis was prompted by a persistent problem encountered in conducting 
methodological research o^n knowledge use and school improvement. This problem, 
in its broadest terms, stems from the dearth of available procedures to map the 
frames of reference of stallholders in school improvement such that we may 
reasonably claim that they belong to an identifiable "epistemic community" ( s ®e 
Holzner and Marx, 1979; Dunn and Holzner, 1981). \ t 

We are currently developing /procedures to elicit and identify the frames 
of reference of individuals (see Mclntyre, Fisher, and Dunn, 1981; Dukes and 
Dunn, 1981; Dunn and Dukes, 1981). Nevertheless, we also require an appropriate 

v. 

conceptual framework and methodology to characterize the frames of reference of 
collectivities . We believe that social network analysis provides a basis for 
developing one such conceptual framework and methodology. ^ 

The importance of devising appropriate procedures for studying knowledge 
use as a collective phenomenon is evident once w^ abandon individualistic and 
mechanistic assumptions about the role of knowledge in policymaking and prof^s- 
sional practice. Individuals do not make binding policy decisions; they rather 
share decisional power and influence with multiple stakeholders whose divergent 
needs and interests create policies of great collective complexity (see Weiss 
and Bucuvalas, 1980). Under such conditions a narrow "problem-solving" image 
of knowledge use and school improvement is particularly inappropriate, mis- 
leading, and inapplicable (Cohen and Ganet, 1975). Moreover, knowledge is not 

"product-embodied," in the sense that particula^research findings, technolo- 

\ * J 

gies, or accumulated rules of successful practice feraft- knowledge) are 
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mechanically " transf erred 11 or "linked 11 to others. Knowledge is rather a social 

construction which is transacted by multiple stakeholders with different frames 

> m 

. of reference. x 
s 

Thus, "knowledge use 11 includes but goes beyond social interaction and 
exchange; it involves communicative actions between two or more stakeholders 
whos^relation to one another, and to some educational innovation, is mediated 
by their respective frames of reference. Processes of knowledge use are there- 
fore interpretive and interactive at once, such that "cognitive" and "social" 

dimensions of knowledge use may be separated analytically but not in practice 

< 

(compare House, 1981). For this reason school improvement and other types of 
social reform are best viewed as "critical social transactions" among different 
stakeholders who together Negotiate the adequacy, relevance^ and cogency of 
knowledge claims (Dunn, 1981). 

The major conclusion of tKis paper is that Appropriate modifications and 

I 

extensions of social network analysis will enable investigators to map the 
complexity of knowledge transactions. Specifically, by studying "knowledge 
networks" we can: <(1) delineate epistemic communities whose key property is 

the sharing of a common frame of reference towards knowledge use and school 

1 V> 

improvement; (2) monitor changes in the frames of reference of epf^temic comm- 

\ 

unities; and (3) discover relationships between patterned social interactions 
(social structures) and changes in the frames of references and boundaries of 
epistemic communities. 

In arguing towards this conclusion this paper first outlines conceptual 
,and methodological bases of social network analysis, including theoretical 
perspectives that originate in Afunctional ism and exchange theory and methods of 
' procedure that lie in the tradition of sociometry. The second part of the paper 
Reviews relevant past, applications of social network analysis In 3everal domains 
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including energy policy, corporate ownership and control, community power struc- 
ture, and the diffusion of innovations. Finally, the paper proposes and justifies 
the idea of "epistemic network analysis" as an appropriate and useful addition to 
research on social networks in general and to studies of knowledge use in parti- 
cular. • 

. ; • BASIC CONCEPTS AND METHODS 

In its most basic and general form a social network may be defined as "a 
specific set of linkages among a defined set of persons with the addit^nal ' 
property that the characteristics of these linkages as a whol6 may be used to 
interpret the social behavior of the persons involved" (Mitchell, 1969). This 
definition, which focuses on linkages among persons, may be extended to include 
collectivities. -Thus, Lauman and Pappi (1976) define a social network as "a 
set of nodes (e.,g., persons) linked by a set of social relationships of a 
I specified type." This definition, since it is generalizable tb any and all 
units of analysis (inaividuals, dyads, eliques, elites^ groups, organizations, 
communities) , permits us to study collective properties of knowledge properties 

and use. Moreover, .this, definition stipulates that diverse social relationships, 

' • t * 

. and not .linkages per se, characterize the interactions among individual and 

# 

collective actors. These diverse social relationships are' not confined to 
observable exchanges of goods, services, or information but extend to beliefs, 
values, arid assumptions as well. 

Social network analysis originates in two important but disparate streams 

t 

of theorizing about social phenomena. The first, represented by structural 
sociology, is exemplified by theories of exchange structuralism, functional 
structuralism, and conflict structuralism (see Wallace, 1970). Here network 
analysis has. been employed to mode], those sets of components, interactions, and 
Relationships which together constitute "social structure. " For example, network 
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a n a l ysis has been used both to operatidnalize and develop structural-function- 
alism (Lauman and Pappi, 1976; McCord, 1980). f *" ■ 

A second stream of theorizing which underlies network analysis is mathe- 
matical (see, e.g., Harary et al., 1965). Here we find the use of both geometric 
and algebraic models to represent social relationships. Graph theory, as a 
combination of geometric and algebraic concepts and operations, provides one of 
the foundations of network analysis. Graph theory not only permits us to 
represent social networks in the form of directed and undirected graphs; it also 
enables the translation of graphic representations into matrix form in order to 
perform algebraic operations. Following graph theory, network analysis also 
.permits the assignment of values to loops and lines. 

There are two basic types of networks: systems networks and system-state 
networks (see Heckathorn, 197*9). Systems networks focus oa^structure and poten- 
tial social relationships, while system-state networks deal with process and 
actual social relationships. For example, a systems network might assume the. 
specific form of a professional teachers association whose structural proper- 
ties and potential for change may be described in general. By* conp£ast , a 
system-state network might be found in the form of a teachers 1 union engaged in 
processes of bargaining and conflict with district school administrators. 

Formally, a systems network may be defined as a set of no^es (e.g., teachers) 

/ f ^ -7 

connected by arcs representing the potential transfer of W valued or disvalued 
good or service (e.g., curricular materials). A system-state network, by 
contrast, represents' an actual rather than potential transfer. For example, 0 
while curricular materials are potentially tr^nsf errable among teachers the 
actual transfer of such\^goods" may be highly segmented. t The concepts of 
systems network and system-state network are useful in characterizing properties 
of different social structures. 1 

Properties of social structures may also be investigated with blockmodeling, 
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a set of techniques based upon network procedures (see Lauman and Pappi, 1976; 
Light and Mullins, 1979). Blockmodeling, first introduced in the mid-1970s 
(White €it al. , 1976) , enables investigators to partition units of a social system 
into "structurally equivalent" subgroups (blocks) that are based on patterned 
social relationships. V A blockmodel analysis usually begins with a set of k 
sociomatrices that represents the patterns formed by k different kinds of ties 
among n people or units. The units, if they are not persons, may be stake- 
holder groups, organizations, or communities. The resultant set of blocks may 
be viewed as representing the range of Social roles, positions, or statuses 
present in a system. Thus, blockmodeling partitions a population into sub- 
groups (blocks) while preserving the totality of the system. 

Many concepts and procedures of network analysis are essentially static 
since they * represent structural properties of social systems at single points 
in time. In response to this limitation Holland and Leinhardt (1977) have a 
developed dynamic network model. This model uses continuous-time Markov pro- 
cesses to depict structural change, with structure "conceptualized as influen- 
cing or constraining individual behavior through its effect on the probability 
that a given link (or non-link) between two individuals will change over time." 
The dynamic model views structural change as continuous, thus avoiding distor- 
tions that arise when networks are investigated at discrete moments. 

The first step in conducting a network analysis is to specify appropriate 
individual and/or collective units of analysis. The second step is selecting 
an appropriate sampling procedure, including forms of non-probability sampling 
that depart from or modify the equal-probability-oir-selection method (EPSEM) . 
For example, network analysis may require interviews with all eligible respon- 
dents in a system, or a sample of class of such systfems, rather than a standard 
random sample of scattered individuals in a large population (Rogers, 1979). 
The choice of a sampling procedure depends both upon the unit of analysis- 
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(individual vs, collective) and the underlying theories (cognitive vs, behavioral) 
^employed to specify and explain social relationships/ Sampling procedures may 
assume a number of different forms that modify or depart entirely from assump- 
tions of probability sampling (see Kish, 1965). These modifications or depar- 
tures include theoretical sampling (Glaser and Strauss, 1967), sociometric 
sampling (Sudman, 1976), stakeholder sampling (Dunn, Mitroff, and Deutsch, 19S1) , 
and sampling procedures oriented towards the selection of issues, events, or 
phases in the process of planned change and knowledge use (Dunn and Holzner, 
1981), 

Thus far, network analysis has relied primarily on data obtained from 
responses to sociometric questions included in self-administered questionnaires 
and interview schedules. Sociometric question? may focus on how individuals 
feel towards one another (Hunter, 1978), who meets whom about what on a regular 
basis (Zijlstra, 1979), or the sources of information leading to the adoption 
of innovations (Rogers, 1979), The choice of particular types of questions 
implies the selection of appropriate measures and measurement criteria. 
Examples include the amount of communication, as measured by the length of time 
and number of actors involved; the perceived importance and content of communi- 
cation; frequency, content, and intensity of interaction; and cognitive, affec- 
tive, or ideological distance. 

The reliability and validity of ^ociometrically based network studies have 
been questioned by several commentators- For example, Kilworth (1976) and 
Bernard (1977) have cautioned that four considerations must be taken into 
account if we are to* assess the error within instruments used to collect socio- 
metric or network data: reliability, validity, veri^tlality , and accuracy. In 
this context there is some evidence that cognitive H#ta are inaccurate about 50 
percent of the time and, for this reason, cannot be used to represent social 
structure and behavior (Bernard, Killworth, and Sailer, 1979), One solution for 
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this problem is to employ naturalistic obseryation, as opposed to cognitive data, 
to describe behavior. Another solution is to increase the veridicality of cogni- 
tive data by employing various qualitative_procedur;es (see Mclntyre, Fisher, and 
Dunn, 1981), including instruments that have proved effective in other contexts 
of ^actice (e.g., marketing) as jneans for uncovering personal constructs and 
assumptions (see Frost and Braine, 1967; Dunn and Dukes, 1981). Finally, it 
should be noted that standard reliability and validity issues may be somewhat 
beside the point if we begin from the assumption the cognitive data are idio- 
graphic and dynamic, thus rendering standard psychometric validation procedures 
(e.g., internal consistency and test-retest reliability) in appropriate and 
inapplicable (see Kelly, 1955). 

APPLICATIONS 

Network analysis has been applied in a variety of contexts. Applications 
of network techniques, as Holland and Leinhardt (1979) observe, have^sttufcd 
from studies of attitude formation and communications to research on social 
mobility, interpersonal and political behavior, corporate and community organ- 
ization, and the diffusion of innovations. 

Some of the most successful applications of network analysis have been in 
the area of research on the diffusion of innovations. Whereas the bulk of 
existing research on innovation, planned change, and knowledge use focuses on 
individuals, network analysis offers possibilities for studying the relational, 
transactional, and collective properties of knowledge creation, transfer, and 
use (see Lazarsfeld, Pasanella, and Rosenberg, 1972: Section III; Coleman, 
Katz, and Menzel, 1957; Coleman, 1958-59; Rogers, 1979). The relational, 
transactional, and collective aspects of social systems are critical to the 
diffusion of new ideas, products, and practices as well as to the utilization 
of knowledge in general. 
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^ An instructive early example of network analysis is Coleman, Katz, and 
Menzel (1957), a study of the social processes affecting the adoption of a new 
drug by physicians. Doctors in four communities were interviewed, using socio- 
metric questions, and grouped into seven subgroups (cliques). In Coleman's 
words : 

The question, then, became this: At each point in time after 
the drug was n^arketed, Were cliques homogeneous or not in 
their members' use or nonuse of the drug? If they were 
homogeneous, then this was evidence that some kind of social 
influence pr diffusion was going on in relation to the mea- 
sured sociometric ties. If not, this indicated that the 
cliques delineated on the basis of questions in the interview 
had little relevance to drug adoption (Coleman, 1958-59:33). 

In monitoring the adoption process over a 15-month period Coleman and 
colleagues found that there wa's no subgroup homogeneity until around 7 ^months 
after the drug was introduced, ar time at which some 50 percent of individual 
doctors had adopted the drug. The maximum homogeneity itaa reached at abtfut 
11 months, a time at which some 76 percent of the doctors had adopted the drug. 
At the end of the 15-month * period 86 percent of doctors had adopted the drug*, 
yet the homogeneity of cliques had returned to zero. The major conclusions 
inferred from these results are that social networks are effective media for 
diffusing innovations at certain times, but not at others, and that social 
network analysis is a powerful method for delineating the changing boundaries 
of collectivities whose individual members are engaged in knowledge transactions. 

In another more recent application of network analysis to the study of 
innovation Butt (1980) has argued that the act of adopting an innovation re- 
fleets a particular type of structural interest. For a potential , adopter the 
perceived utility of an innovation i^ as much a function of his relationship 
with other potential adopters (i.e., his links to a%articular social structure) 
as it is a consequence of external material properties of an innovation, or what 
Capiat! et al. (1975) term "knowledge-specific" characteristics. In the case of 
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medical innovations Burt suggests that a doctor's prescribing a new drjjg is not 
merely a function of his personal characteristics, but also a consequence of his 
position in a social network. This conclusion reinforces findings in other con- 
texts, including mental health, that social relationships based on interpersonal 

» 

-trust are associated With the utilization of evaluation research (see Patton et 
al. , 1977). 

Network analysis also provides a basis f£>r examining the structural rela- 
tions and interactions among actors involved in different phases of the policy- 
making process. By focusing on interorganizational communications we can assess 
the contribution and influence of each policy actor (Zijlstra, 1978—1979), an 
approach that yields information about networks of interlocking directorates 
(Zijlstra, 1978-1979; Levine and Roy, 1979). This type of network analysis 
provides information about representation and control, information access and 
exchange, and negotiation, coordination, and harmonization of interests. Thus 

far, interlocking directorates have been studied in contexts of nuclear energy 

4t 

policy in the Netherlands and decision making among major corporations in the 
United States. , * 

EPISTEMIC NETWORK ANALYSIS 

A major *aim of current and future research projects is to develop appro- 
priate methodologies! for analyzing knowledge transactions among a diverse set 
of stakeholders in school improvement. Knowledge transactions, which represent 
one type of social interaction, may be represented in terms both of structure 
and process. Thus, for example, we can speak of the cognitive structure of 
epistemic communities, a structure constituted by patterns of express and 
tacit commitments to particular tests for assessing the adequacy, relevance, 
and cogency of knowledge claims (Dunn and Holzner, 1981). 

We can also investigate the social processes that lead to the creation, 
maintenance, and transformation of the cognitive structures of epistemic* 
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communities. In focusing on such structures and processes we employ argument 

as a unifying construct that encompasses social as well as cognitive properties 

of social interaction (Dunn, 1981), In using argument as a model of knowledge 

production and use we assume that the structure of epistemic communities is a 

product of natural social comparison processes which may be investigated 

empirically. Arguments, as Willard observes, are 

naturally occurring corollaries to research contexts... It is 
rather as if we were to stand back and watch while our subjects 
framed their own hypotheses, selected methodological principles, 
utilized techniques appropriate to both, and conducted their own 
research act. . .arguments give us more information than other 
kinds of research (Willard, 1980:9-10). 

The structural model of argument developed by Toulmin (1958) provides a 
useful framework for investigating knowledge transactions. Knowledge transac- 
tions affect and in turn are affected by the frames of reference of individual 

J 

and collective stakeholders. In order to deal with the complexity of knowledge 
transactions an4 their effects on the production and uSe of knowledge in con- 
texts of educational practice it is necessary to characterize and bound the 
cognitive maps of stakeholders. Given that network analysis may be readily 
adapted to problems of cognitive mapping (Heckathorn, 1979), what is needed is 

a kind of "epistemic network analysis." 

f. 

Epistemic network analysis, as the term suggests, begins with the idea of 
"epistemic community" as a focal point. An epistemic community, translated 
into network terms, may be defined as a set of nodes, represented by collective 
and individual ictors, linked by a set of* social relationships that are based 
on the acceptance of a common frame of reference. A frame of reference may be 
defined as a particular configuration of explicit and implicit standards for 
affirming or challenging the adequacy, relevance, and cogency of knowledge 
claims, whether "scientific" or "ordinary" in origin (cp. Lindblom and Cohen, 
1979). 

1 Tc 
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In studying epistemic or knowledge networks we may transform and extend the 
structural model of argument to represent major elements of social discourse and 
"debate. Specifically, new ideas, practices, or products constitute data (D) on 
the basis of which educational stakeholders offer knowledge claims (C) . Know- 
ledge claims may be definitive ("Standardized achievement tests are objective"), 
designative ("The socioeconomic status of students governs their educational 
achievement"), evaluative {"Instructional television is bad for children"), or 
advocative ("School districts should adopt busing as a means to integrate 
schools") . 

The structural model of argument provides that data (D) are transformed 
into claims (C) via warrants (W) and backings (B) for warrants. Warrants 
answer the question "Why does a knowledge claim follow from the data supplied?" 
by providing plausible reasons. For example, "Since District A is essentially 
similar to District B" (W) , and "District B achieved racial balance through 
busing" (D) , then "District A should adopt busing as a means to integrate schools" 
(C) . A backing (B) for the, warrant described above answers the question of why 
we should accept the warrant. In the example just offered the backing (B) might 
be "Because District A and District B haye similar ethnic and socioeconomic 
profiles." The backing (B) and warrant (W) for a claim constitute part of what 
is typically meant by the term "frame of reference" (Dunn and Holzn^, 1981). 

The last key element in the structural model is the rebuttal (R).* The 
rebuttal is a counterclaim, counterwarrant , or counterbacking offered to dis- 
pute or challenge another stakeholder. In effect, the rebuttal establishes the 
social character of an argument, since all knowledge claimants are linked to 

*For purposes of exposition we have omitted Toulmin's qualifier (Q) , which 
expresses the relative force or plausibility of a claim. Hie qualifier may 
be used to operationalize the concept of "cogency test" .(see Dunn, 1981). 
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counter-claimants in a network or field of discourse and debate. By definition, 
then, argument is a social comparison process involving claims and counterclaims 
about the adequacy, relevance, and cogency of new educational ideas, practices, 
and products. 

The above discussion suggests two major types of epistemic networks: 
manifest and latent . Manifest networks are composed of claims and counterclaims, 
that is, overt and publicly available statements about the nature, consequences, 
value, or action- implications of an innovation. These claims and counterclaims^ 

are similar to responses in public opinion surveys. At another more profound 

> 

level are networks of warrants and backings that together constitute frames of 
reference. These latent networks embody the unspoken or hidden standards of 
assessment operative in affirming or contesting knowledge claims, 

Epistemic network analysis may be used to monitor changes or transformations 
in manifest or latent networks. Two available methods for monitoring such trans- 
formations were noted in the first section of this paper. One method focuses on 
a temporally ordered system state newtork, while the other operationalizes net- 
work dynamics as a Markov chain. Although transformations in manifest and 
latent networks may be continuous or discontinuous (as in "catastrophe theory") , 
it is reasonable to represent changes by comparing manifest and latent system 
states at different points in time. In this way we may uncover those elusive 
properties of "collective learning" and "knowledge synthesis" that are so criti- 
cal to research on knowledge use and public policy (see Etheredge, 1979; Holzner, 
1979; Rich, 1981). 

In discussing the structure of epistemic networks we have thus far avoided 
any discussion of social interaction. ' Clearly, epistemic networks are based 
both on social interaction among occupants of different social role9 (e.g. , 
parents, teachers, principals, superintendents) as well as the negotiation and 
acceptance of particular frames of reference. Yet the investigation of "social" 
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and "cognitive" properties of epistemic networks requires different procedures. 
Thus, for example, social positions or roles may be Sampled randomly or enumerated, 
while frames of reference require theoretical, purposive, or snowball sampling 
designs. Similarly, social interaction may be investigated with standard socio- 
metric instruments' t-hat focus, for example, on the type, frequency, and inten- 
sity of interaction. By contrast, the discovery of latent (but not manifest) 
epistemic networks requires depth interviews to elicit underlying standards for 
» assessing the adequacy, relevance, and cogency of knowledge claims. 

These contrasting methodological requirements punctuate the importance of 
concurrently studying social and cognitive properties of epistemic networks. 
Procedures for studying social interaction, as well as what we have called 
manifest networks, are reasonably well developed in available research. Pro- 
cedures for studying "cognitive maps," "paradigms," "frames of reference," and 
other latent structures are much less readily available (see Dunn and Dukes, 
1981). fyeeded are research methods that effectively surface tests of knowledge 
adequacy, relevance, and cogency and appropriate conceptual frameworks to order, 
differentiate, and label these varied components of frames of reference. 

CONCLUSIONS 

This paper has described basic concepts and procedures of ^ocial network 
analysis, reviewed several past applications, and proposed and justified the 
idea of "epistemic network analysis." Our major purpose has been to sketch 
the contours of an approach that extends and modifies network analysis in ways 
that permit the systematic study of epistemic communities. Epistemic network 
analysis, by focusing on manifest and latent knowledge structures, can assist 
in further operationalizing the concept of frame of reference, establishing on 
empirical grounds the boundaries of epistemic communities, monitoring changes 
in the manifest and latent knowledge structures, and describing relations between 
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the dynamics of social interaction and changes in individual and collective 
frames of reference. Epistemic network analysis may even permit systematic 
investigation^ of collective learning and knowledge synthesis, processes that 
so far have eluded mo$t students of knowledge use and professional practice. 



\ 



ERIC 17 1, 



REFERENCES r 



BARNES, J. A. (1979) "Network analysis: OrieftTing notion, rigorous 
technique, or substantive field of study," in Perspectives on Social Network 
Research, Holland and Leinhardt (eds.). New York: Academic Press. 

BERNARD, H.R. et al. (1979) "Informant accuracy in social network data," 
in Social Networks, Vol. 2, No. 3. 

BERNARD, H.R. and P.D. KILLWORTH (1977) "Informant accuracy in school 
network data II," in Human Communication Research, Vol.* 4, No. 1:3-18. 

BOISSEVAIN, J. and J.C. MITCHELL (1973) Network Analysis Studied in 
Human Interaction. The Hague: Mouton & Company. 

9 

BREIGER, R.L. (1979) "Toward an operational theory of community elite 
structures," Quality and Quantity, Vol. l5: 21-57. 

BURT, R.S. (1980) "Innovation as a structural interest: Rethinking the 
impact of network position on innovation adoption," Social Networks, 2:327-355. 

CAPLAN, N. , MORRISON, A., and R.J. STAMBAUGH (1975) The Use of Social 
Science Research in Policy Decisions at the National Level. Ann Arbor: LSR- 
CRUSK, University of Michigan. 

CARTWRIGHT, D. and F. HAVARY (1979) "Balance and clusterability : An. 
overview," in Perspectives on Social Network Research. Holland and Lein- 
hardt (eds.). New York: Academic Press. 

COHEN, D.K. and M.S. GARET (1975) "Reforming educational policy with, 
applied social research," Harvard Educational Review 45, 1:17-43. * 

COLEMAN, J.S. (1958) "Relational analysis: A study of social organi- 
zation with survey methods," Human Organization, 17:28-36. ^ 

COLEMAN, J., et al. (1957) "The diffusion of an innovation among physi- 
cians," Sociometry, Vol. 20:253-270. 

DUKES, M.J. and W.N. DUNN (1981) "Psychometric procedures for research 
on knowledge use and school improvement." Working Paper KU-07. Pittsburgh: 
University Program for the Study of Knowledge Use, University of Pittsburgh. 

DUNN, W.N. (1981) "Reforms as arguments," Knowledge: Creation, Diffu- 
sion, Utilization, 3, 1 (forthcoming). 

DUNN, W.N. and M.J. DUKES (1981) "Multiattribute procedures for research 
on knowledge use and school improvement." Working Paper KU-08. Pittsburgh: 
University Program for the Study of Knowledge Use, University of Pittsburgh. 

DUNN, W.N. and B. HOLZNER (1981) "Methodological research on knowledge 
use and school improvemerit : Conceptual framework and study design." Working 
Paper KU-01. Pittsburgh: Program for the Study of Knowledge Use, University 
of Pittsburgh. ^ 



ERLC 



1 



7 



DUNN, W.N., MITROFF and S.J. DEUTSCH (1981) "The obsolescehce of 
evaluation research," Evaluation and Program Planning 5, 1 (forthcoming). 

• ETHEREDGE, L.S. (1979) "Government learning: An overview." Cambridge: 
tenter for International Studies, Massachusetts Institute of Technology. 

^ * FRANK, 0. (1978) "Sampling and estimation in large social networks," 
Social Networks, 1:91-101. 

FROST, W.A.K. and R.'L. BRAINE (1967) M The application of the repertory 
grid technique to problems in mafcket research," Commentary 9, 3:161^175. 

GLASER, B. and A. STRAUSS (1967) The Discovery of Grounded TheoVy. 
Chicago: Aldine. 

* 

HARARY, F. et al . (L965) Structural Models: An Introduction to the 
Theory of Directed Graphs. New York: John Wiley and Sons, Inc. 

iHECKATHORN , D. (1979) "The anatomy of social/network linkage*," Social 
Science Research, Vol. 8:222-252. » 

HOLLAND, P.W. and S. LEINHARDT (1979) Perspectives on Social Network 
Research. New York: Academic Press. 

HOLLAND, P.W. and S. LEINHARDT (1977) Dynamic model for social net- 
works," Journal of Mathematical Sociology, Vol. 5:5-20. 

HOLZNER, B. (1979) "Social processes and knowledge synthesis." Occa- 
sional Paper KU-109. Pittsburgh: University Program for the Study of 
Knowledge Use, University of Pittsburgh. 

HOLZNER, B. and J. MARX (1979) Knowledge Application: The Knowledge 
System in Society. Boston: Allyn and Bacon. 

HOUSE, E. (1981) "Three perspectives oi^ innovation: Technological, \ 
political, cultural," . in R. Lehming and M. Kane (eds.), Improving Schools* 
Using What We Know. Beverly Hills: Sage Publications. ' 

HUNTER, J.E. (1978) "Dynamic sociometry," Mathematical Sociology. Vol. 
6:87-138. 

KELLY, G.A. (1955) The Psychology of Personal Constructs, \ Vols. New 
York: Nortpn. 4 m * 

KILLWORTH, P.D. and H.R. BERNARD (1976) "Informant accuracy in social 
network data," Human Organization, Vol 35, No. 3:264-285. 

KISH, L. (1965) Survey Sampling. New York: Wiley. 

LAUMAN, E.O. and F.U. PAPPI (1976) Networks of Collective Action: A 
Perspective on Community Influence Systems. New York: Academic Press. 

LAZARSFELD, P., PASANELLA, A. and M. ROSENBERG (1972) Continuities in 
the Language of Social Research. New York: Free Press. 



erJc 1 ?\, 



LEVINE, J.Hv and W.S. ROY (1979) "A st^y of -interlocking directorates: 
Vital concepts of organization, 11 in Perspectives on Social Network Research, 
Holland and Leinhardt (eds.) . New York: Academic Press, - 

LIGHT, J.M. and N.C. MULLIJIS (1979) "A primer on blockmodelirig procedure, 11 
in Perspectives on Social Network Research, Holland and Leihtiardt (eds.). New 
York: Academic Press ^ 

LINDBLOM, C.E. and D.K. COHEN (1979) Usable Knowledge: Social Science 
and Social Problem Sojvingv New Haven: Yale University' Press. 

MCCORlD, E. (1980) "Structural-functionalism and the natwork idea: Towards 
an integrated methodology," Social Networks, 2:371-383. \^ 

MCINTYRE, T«, FISHER, E. an<J W.N. DUNN (1981) "Qualitative procedures for 
^research on knowledge* use and school improvement.. M Working Paper KU-06. 
Pittsburgh: University Program for the Study of Knowledge Use, University of 
Pittsburgh. 

t ■ ** 

fclTCHELL, J.C. (1969) Social Networks in Urban Situations. Manchester: 
Manchester University Press. , 

PATTON, M.Q. et al. (1977) "In search of impact: An analysis of the 
utilization of federal health evaluation research," in Using Social Research 
in Public Policy Making, (ed.) C.H. Weiss. Lexington: D.C. Heath. 

.RICH, R.F. (1981) "Knowledge sy thesis and problem solving." Discussion 
draft. • i 

ROGERS, E.M. (1979) "Network analysis of the diffusion of innovations," 
in Perspectives on Social Network Research, Holland and Leinhardt £eds.)..' 
New York: Academic Press. 

J SANGER, R. (1978) "A network method and its uses in urban ethnography," - 
Human Organization, Vol. 37, No. 3-: 257-268, Fall. 

SUDMAN,, S. (1976) Applied Sampling. New York: Academic Press. 

WEISS, C. and M. BUCUVALAS (1980) Social Science Research and Decision 
Making. New York: Columbia University Press. 

WHITE, H.C. et al. (1976) "Social structure from multiple networks. /I. 
Blockmodels of roles and positions," American Journal of Sociology, Vol. 81 
So. 4:730-730. 

WltLARD, C.A, (1980) A Theory of Argumentation. Unpublished Manuscript. 



ZIJLSTR** G.J. (1978/9) "Networks in public policy: Nuclear energy in 
the Netherlands/' Social Networks, 1:359-389. ^ 



17 j 



VI MULTI ATTRIBUTE PROCEDURES FQR 

STUDYING ADOPTION DECISION MAKING 



William N. Dunn and Mary Jo Dukes 



Introduction 

The Multiattribute Problem 

Types of Multiattribute Procedures 

Selection of Attributes 

—Stimulus-Imposed 

—Response- Imposed 

—Stimulus and Response- Imposed 
Explanation of Judgments ? 
Unit of Analysis \ 
The Judgment Process ♦ 
Conclusions 



16 



u 



ABSTRACT 



A major problem 1n decisions to adopt an educational 
innovation, whether as an Individual or as a committee 
member acting on behalf of a larger unit, is how to 
assess the Innovations available. This paper suggests 
thatthe choice of a subset of usable measures of 
assessment is a multl attribute problem, that is, a 
problem where the selection of an appropriate subset 
4 depends on the subjective standards of assessment 
applied by the stakeholders w"ho attribute different 
properties to the Innovation; compares and contrasts 
/ multl attribute procedures/ according to several key 
j dimensions, including the mode, scope, ground, and focus* 
of procedures for selecting assessment measures; and 
evaluates these procedures in terms both of their 
general appropriateness to complex problems of choice* 
and their applicability to .the specific problem of 
selecting a subset of maximally usable measures to assess 
the potential of the Innovation. 
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INTRODUCTION ' u 

\ 

In this paper we explore the potential applications of multi attribute > 
decision theory to knowledge use and school improvement. The decision to 

use, adapt or reject an innovation 1s a complex phenomenon. One of the 
primary reasons for, this complexity 1s *that knowledge use is fundamentally 
interpretive. In assessing the utility of an innovation, indiivi duals, 
depending uj>on their frame of reference, attribute different properties or 

attributes to different innovations'. The various attributes are not 
usually weighted equally in importance by all stakeholders. 

This paper assumes that a judgment regarding the utility of an 
educational innovation is best viewed as a multiattribute problem—that ' 
is, a problem where the overall assessment of the utility of an innovation 
depends on*the subjective standards of assessment or decision rules used 
by stakeholders who attribute different properties to educational Innovations. 
This and similar multiattribute problems are the focus of an extensive and 

a 

highly dlverslfi^fMiterature that may be characterized by the term 
multiattribute decision theory; Nevertheless, it should be recognized that 
this umbrella term Includes approaches, models, and procedures that differ 
fundamentally in their aims, assumptions, and contexts of application. 
Thus, for example, while evaluation researchers generally equate multi- 
attribute procedures with the work of Keeney, (1975), Keeney ?nd Raiffa (197$)', 
and Edwards, Guttentag, and Snapper (1975), this particular variant of 
multiattribute decision theory 1s an extension of "axomatic utility theory 



(see von Neumann and Morgenstern, 1947) that has little 1n common with 
other fVrms of multi attribute decision theory. These other forms of multi - 
attrlbute decision XXptory include social judgment theory (Hammond, 1965), 
conjoint measurement (Krantz and Tversky, 1971), functional measurement 
(Anderson, 1970), analytic hierarchy analysis (Saaty, 1980), multidimensional 
scaling (Kurskal and Wish, 1978)^ strategic assumption*! ^analysis (Mitroff , 
Emshoff, and Kilmann, 1979), and dialectical performance assessment 
(Dunn, Mitroff, and Deutsch, 1981). 

We thus use the term multi attribute decision theory as a broad 
methodological construct that includes many diverse procedures. We have 
deliberately assumed a flexible mult1d1sc1pl1nary posture towards these 
procedures since we believe that the discipline and research traditions 
with which they are associated often create artificial boundaries that 
limit potential applications to evaluation research, knowledge use, 
innovation adoption, and other areas of policy research. Our aim 1s to 
select elements of these various multiattrlbute procedures on the basis 
of their potential contribution to understanding judgments regarding the / 
utility of innovations rather than their conformity to the research 
traditions from which they arose. To borrow and adapt the well-%iown contrast 
between "trapped" and ^experimental" administrators (Campbell, 1975), we 
view our task as one of advocating procedures on the basis of the severity 
of the problem rather than the certainty that any one procedure will yield 
conclusive and final results. We thus see our role as experimental 

methodologists, 1n the widest sense of the term, rather than trapped ones. 

Q 

The purpose of this working paper is threefold. First, we present the 

* f" 

general case for viewing educational innovation assessment as a multiattrlbute 
problem. In making this case we rely on analogies to marketing, a field 
of research and practice whose aims, assumptions, and problems bear a close 

Is- 
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resemblance to those encountered in the development, testing, and diffusion 
of educational innovations. Second, we compare and contrast muUiattribute 
procedures according to several dimensions that are of criticaKimportamce 
1n identifying a subset of criteria used to assess the utility of educa- 
tional innovations. By considering these dimensions— which include the 
mode, scope, ground, and focus of procedures for analyzing judgments of 
the utility of educational innovations— we are able to address key 
methodological and practical issues and avoid surface technical disputes, 
for example, how best to estimate the shapes of utility functions or j 
whether to use full or partial factorial designs. Finally, we evaluate these 
procedures in terms both of their general appropriateness to iVl-struitured 
problems of choice and their applicability to the specific problem at hand- 
that 1s, Identifying a subset of Innovation attributes used to assess 
, educational Innovations and determining the relative Importance of the j 
Innovation attributes with regard to judgments of the overall utility of t 
Innovations. 

THE^MULTIATTRIBUTE PROBLEM 

Identifying Innovation attributes and determining how these attributes 
are combined to arrive at a judgment of the utility of the Innovation 1s 
a muUiattribute problem, By this we mean that understanding judgments 
regarding the usefulness of an educational Innovation depends on our 
capacity to uncover and model the subjective standards of assessment or 
decision rules employed by stakeholders who attribute conflicting properties 
to educational Innovations. Some of the most relevant characterizations of 
this problem come from fields of marketing (e.g*, Green and Wind, 1973), 
Information processing (e.g., Slovic and Lichtensteln, 1971), and manage- •? 
ment Information systems (e.g. , Larcker and Lesslg, 1980), where -problems 

f 
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of assessing the usability of ^products anc j processes are highly dependent 
on pur understanding of the subjective judgments of consumers and users. 

This same problem, at least 1n Its most general form, 1s also the 
prlnkary concern of that branch of research utilization that is based on an 
extension and adaptation of the classical sociology of knowledge (Holzner 
and Marx, 1979>. Here we find that the classical sociology of knowledge, 
with Its emphasis on the social determinants of Intellectual formations, 
is "turned upside down" and thereby transformed into a social science of 
knowledge applications (Weiss and Bucuvalas, 1980). The central unifying^ 
construct of a social .science of knowledge applications is the Idea of 
''frame of reference" (Holzner and Fishery 1979), a construct that denotes 
systematically organized sets of underlying principles, rules, or tests 
that shape decisions to accept, reject, or modify products and processes of 
applied social research. Dunn U981) has developed a preliminary cla'ssl-- 

'fir 5 

fi cation of such tests and proposed, 1n contrast to Campbell and Stanley 
(1963), a series of threats to the adequacy, relevance, and cogency of 
usable knowledge. 

In viewing judgments about the utility of educational innovations as 

a multiattrlbut^ problem we assume that the dimensions of the Innovation, 

( 

can be described in terms of states or levels of attributes that 

originate in the underlying assumptions of stakeholders. As Green and Wind 

observe, attributes a|e 

mental percepts and may or may not be related to object- 
ive characteristics of the object. The attributes that 
are evoked are assumed to be related to the decision 
maker's knowledge of and experience with the objects 
and the characteristics of his Specific problem-solving 
situation (Green and Wind, 1973:14). 

In assessing the utility of educational innovations and other areas, 

attributes may be structural (e.g., teacher feedback system), functional 

i 

- ( 

1 S , 



(e.g., the extent to which the Innovation will Increase the attainment of 
desired outcomes), psychological (e.g., how the Innovation may affect 
professional self-image), social (e.g., the status conferred on the school 
through adoption of an innovation), or economic (e.g., the financial costs 
of implementing and maintaining a particular innovation). 

For present purpo%fes we can formally represent objects of judgment— 
whether these be approaches to outcome measures or constituent concepts, 
strategies or processes— as discrete alternatives whose selection is 
related to judgments about the usability of an innovation in specific 
contexts of use. Any alternative, X, may be represented as the ordered 
m-tuple 

'X = (Xj, *3* *nj) (1) 
where X is the alternative and Xj, x 2 , x 3 , ... x^ refer to the state or 
level of the j-th attribute (j = 1, 2, 3, n). If we are dealing with 
an attribute state (e.g., [ a curriculum constraint specifying basic 
skills or subject area requirements])the attribute is an unordered dichotomy 
or polytopy that may be coded in term? of p-1 dymmy variables, .where p 
is the number of unordered (nominal) categories. If we are dealing with 
an attribute level (e.g., the perceived decisional relevance of art in- 
structional or management strategy or the dollar costs of implementing 
the ^innovation) the attribute is a continuous variable that may have ^ 
properties of ordinal, interval, or ratto scales*. Assuming that a given 
stakeholder assesses the usability of some innovation as greater than 
others we can represent his usability -function as 

4 U(X) = f(Xj, x 2 , x 3 , ... xj (2) 

where'U(X) is the usability (utility, value, worth) of the X-th alternative 
and Xj, Xg> x.j,...x m are states or levels of attributes tha^ constitute a 



usability function. Usability functions may be linear or configural^dditive 
or multiplicative. ' 

The primary aim of 1nvest1gat1ng^jtia^mejTts^of the utility of educational 
Innovations as a multl attribute problem is to construct models of stake- 

i 

holders' usability functions that satisfactorily account for their choice 
of concepts, strategies, processes, and outcome measures. If the plausi- 
bility of such models can be established through experimental research we 
might also predict the subsequent behavior of stakeholders in using Inform- 
ation supplied with particular innovations. Thus, for example, it would 
be useful to know the underlying subjective basis (usability function) 
according to which an educational stakeholder might weight and combine 
attributes 1n order to choose among the three types of Innovations de- 
scribed in Table 1. 



Innovation 



Table 1 

ATTRIBUTES OF THREE INNOVATIONS 

DESIGNED TO 
INCREASE BASjC «SKILL ACQUISITION 

Attribute 



(1) 

Level of 
Eff prt Required 



(2) 

Goal 
Attainment 



(3) , 

Motivational 
Relevance 



(4) 

Cost per 
School 



Curricular 

Instructional 

Management 



6 
1 
4 



5 
3 
1 



2 
5 
2 




$800 
00 
$300 
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NOTE: Columns (l)-(3) represent arbitrary subjective judgments on a scale 
ranging from 1 to 10. Column (4) contains hypothetical data that 
are, 1n principle, objectively calculable. 

In the above Illustration Innovations and their attributes have been 
simplified for purposes of exposition. Any reasonably complex multi- 
attribute problem will raise at least four key methodological issues 
not addressed in this Illustration: 



V 



(1) Selection of Attributed : What pifecedures are available to select 
attributes such as those listed in Table 1? Is this attribute-apace, 
composed of four elements, smaller or larger than that of given stake- 
holder (e.g., teachers, principals, students, parents, 

•tc.)? To what extent do these four attributes correspond to the real 
but unknown decision rules used by stakeholders to assess the usability ^ 
of educational innovations? These and related questions point to major 
differences in the methodological grounds used to select attributes. 
Generally, these grounds involve attributes that are researcher- imposed 
or user-generated . 

(2) Explanation of Judgments : On what basis" are judgments or choices among T 
alternatives explained? Do regression models that estimate choices on 
the basis ofc linear (or non-linear) combinations of weighted attributes 
provide an adequate basis lor explaining choices among innovation 
alternatives? Do efforts to establish the goodness-of-f it between re- 
gression equations and preestablished formal utility functions and 
axioms provide adequate explanations of choice? These questions accen- 
tuate differences in the theoretical focus of multiattribute procedures. 
Generally, multiattribute procedutLa are as sumpt ion-focused or calibra- 
tion-focused . 

(3) Unit of Analysis ; Do individuals or groups constitute the unit of 
analysis whan applying multiattribute procedures? When individual stake- 
holders are the unit of analysis can we properly aggregate or average 
their judgments \o represent a "collective" choice? When groups are the 
unit of analysis can we describe authentically collective judgment pro- 
cesses, rather than aggregative ones? These questions point to differ- 
ences in the scope of involvement in multiattribute decisions. Generally, 
multiattribute procedures involve segmented or comprehensive involvement. 
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(4) Judgment Process: Do individual or group stakeholders make 

judgments on the basis of their knowledge and experience alone? 
Sr do such stakeholders augment and check their own judgments 
against those of others? These questions call attention to the 
mode of judgment used* to make multlattrlbute decisions. Mode of 
judgment has two general forms that are active and Interactive . 
Most judgments regarding the utility of Innovations 1n the field of 
education. are complex and 111-structured (Dunn and Mltroff, 1980; Dunn, 
Mltroff, and Deutsch, 1981). Consequently, the above Issues niust be addressed 
1n-attempts to devise and apply appropriate multlattrlbute procedures. As 
we shall see 1n the next section these Issues are addressed 1n different ways 
by multlattrlbute procedures that may be compared and contrasted 1n terms 
of the methodological grounds used to select attributes, the theoretical 
focus of explanations, the scope of Involvraertt 1n multlattrlbute decisions, 
and the mode of making judgments about Innovations. 

TYPES OF MULT I ATTRIBUTE PROCEDURES 

In this section we describe six types of multlattrlbute procedures that 
may be employed to model judgments of the utility of educational Innovations. 
These six types are: multlattrlbute utility analysis (MAUT), analytic 
hierarchy process (AHP), conjoint analysis (CONJOINT), functional mea- 
surement (FM), social judgment analysis (SOA), and the dialectical perform- 
ance assessment technique (DIAPAT). The latter procedure was expressly de- 
veloped for purposes of developing Improved performance measures in 
criminal justice but would apply equally well to decisions regarding educa- 
tlonal Innovation adoption, (Dunn, Mltroff, and Deutsch, 1981). 

The procedure followed in this section 1s to address sequentially each 
of the four Issues Identified 1n the last section—selection of attributes, 
explanation of judgments, choice of unit of analysis, and selection of a 
judgment process— by 



comparing and contrasting the six multiat tribute procedures as a set. In other 
words, we compare a^ contrast procedures on an issue-by-issue basis, a procedure 
that maintains a focus on key methodological issues without permitting us to drift 
intd unnecessarily jj|etalled and lengthy technical discussions that are available 

in references appended to this working paper. In addressing these four issues we 

v 

will also apply for purposes of differentiating the six procedures the four 
dichotomous methodological properties specified above — that is, Imposed vs.. 
generated attributes, assumption- yi. calibration-focused explanations, segmen- 
tal vs. comprehensive involvement, and active vs. interactive judgment. These ^ 
dichotomous properties represent our own Judgments about relevant attributes and 
are tied, respectively, to problems of establishing the methodological grounds, 
theoretical focus, scope of involvement, and mode of judgment of multiat tribute 
procedures. These methodological issues, problems, and related properties are 
summarized in Table 2. 



Table 2 



Isdue 



ISSUES, PROBLEMS, AND PROPERTIES 
OF MULTIATTRIBUTE PROCEDURES 

Problem 



Property 



SELECTION OF ATTRIBUTES 



EXPLANATION OF JUDGMENTS 



UNIT OF ANALYSIS 



JUDGMENT PROCESS 



Establish plausible metho- 
dological grounds for 
selecting attributes 

Establish plausible theore- 
tical basis for explaining 
variations in judgments 

Establish plausible basis 
for delineating scope of 
Involvement in multiattri- 
bute decisions 

Establish plausible basis 
for representing judgment 
process f 



User-generated vs. 
re a ear che r- impo s ed 
attributes 

Assumption-focused vs. 

calibration-focused 

explanations 

Segmented vs. compre- 
hensive Involvement 



Active vs. interactive 
Judgment 
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Selection of Attributes 



The problem of establishing plausible methodological grounds for selecting 
attributes is perhaps the key issue in applying multiattribute procedures. In 
every standard application of multiattribute procedures of which ve are aware 
attributes are Imposed by the researcher in one of three major ways:* 

1* Stimulus-Imposed : Here the researcher selects , on a priori, judgmental, 
or intuitive grounds, a set of attributes that are incorporated into 

\ 

standard experimental stimuli (pictures of products, brand flames, vig- 
nettes, scenarios, etc.)* In conjoint analysis (Krantz and Tversky, 
1971) , for example, multilevel factorial designs are used to represent 
various levels of attributes believed to characterize a particular 
object of judgment. Attributes incorporated in standard stimuli are 
then used as a basis for inferring that particular cohflgu rat ions of 
underlying attributes are responsible for patterns of choice among objects. 
2. Response^ Imposed : The researcher again selects attributes on a priori, 
judgmental, or intuitive grounds. But here the Imposed attributes are 
incorporated into response routines (e.g., ranking procedures or Likert- 
< scales) used to Ale it the subject's judgments. In studies of research 
utilization in mental health Weiss and Bucuvalas (1980) , Influenced in 
part by multiattribute utility analysis (Reeney, 1975), use response- 
Imposed attributes whose levels are expressed in five-point Llkert- 
scales, together with randomized standard stimuli in the form of research 
abstracts. Attributes incorporated in response scales are then judged by 
subjects and reduced through factor analysis to sets of factors that con- 
firm or modify original assumptions about the underlying attribute struc- 
ture of respondents. 
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3. Stimulus and Response- Imposed : Here attributes are again imposed on a 
priori, judgmental, or intuitive grounds. But states of levels of attri- 
butes are Incorporated both in standard stimuli (pictures, flash cejrds, 
computer graphics) and. in response routines that typically require subjects 
to rank and weight the importance of predefined attributes that are assumed 
to be related to standard stimuli. Social judgment analysis (Adelman, 
Stewart, and Hammond, 1975) uses stimulus- and response-imposed attri- 
butes in the form of predefined standard stimuli (e.g., descriptions of 
different types of educational research centers) and response routines requiring 

that stimuli be ranked and evaluated in terms of the Importance of prede- 
fined attributes (e.g., scope of change required in Implementation 

of the innovation) to these stimuli. Multiat tribute utility analy- 

sis (Edwards, Guttentag, and Snapper, 1975), while different in most 
other respects, also uses stimulus- and response-Imposed attributes. 
The Imposition of attributes by researchers may seem an efficient or even 
necessary procedure, given the difficulties of eliciting mental constructs from 
respondents. If one assumes that the use of indirect procedures for inferring 
attributes is inevitable, then multiat tribute procedures such a« those derived from 
integration theory (Anderson, 1974) and incorporated as a general heuristic into 
functional measurement routines (e.g., Meyer, Levin, and Louvlere, 1981) will 
suffice. Moreover, if one assumes that particular axioms of choice adequately 
characterize the underlying utility functions of subjects (e.g., Keeney, 1975), 
one can then proceed to establish the goodness-o£-f it between derived axiomatic 

e 

functions and patterns of responses to stimuli. Finally, one can begin from the 
assumption that all persons have Innate feelings on the basis of which they can v 

t 

make consistent judgments that approximate external reality (Saaty, 1980). The 
problem then is to obtain better estimates Of that reality by developing an 
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appropriate mathematical theory for scaling judgments that permits us , to discrimi- 
nate between feelings, establish their consistency with scale-values, and reproduce 
results on repeated occasions (Saaty, 1980:7). , — 

The weakness of these procedures is that they ignore available techniques for 
Identifying, at tributes. Techniques for identifying attributes may be classified 
according to their degree of structure (Hughes, 1971). Relatively unstructured 
techniques include :fca-depth interviews and projective tests, while relatively 
structural techniques include standardized questionnaires and statistical proce- 
dures, associated, for example, with Coombs 1 unfolding theory and Lazarsfeld's 
latent structure analysis (see Coomlfs, 1953). Partially structured techniques, 
to be discussed below, include reason analysis (Lazarsf eld, 1935) .and Kelly f s 
Repertory Grid (Kelly, 1955). As we move from relatively unstructured to rela- 
tively structured techniques we also move along a continuum from user-gener4tad 
to researcher-Imposed attributes. As already noted, the majority of multiat tribute 
procedures Impose attributes by using relatively structured techniques. 
Relatively Unstruc tured Techniques . Unstructured procedures for identifying 
innovation attributes are primarily exploratory in nature, since, the 
content and direction of responses are determined by tfte respondent and not by the 
interviewer. The majority of these procedures have been borrowed and adapted from 
psychology (Collins and. Montgomery, 1970). Relatively unstructured techniques can 
be used with individuals or groups as the units of analysis. 

Depth interviews, which take the individual as the unit of analysis, are con- 
ducted by trained interviewers who probe respondents'- thoughts regarding the use or 
appropriateness of an educational innovation or other object of judgment. Depth 
interviews are unstructured and exploratory in the sense that the course of the 
interview is not determined a priori by the interests of the interviewer. The 
depth interview, however, is not completely unstructured. All interviewers provide 0 
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sane type of feedback to the respondent. This feedback tends to selectively rein- 
force certain kinds of responses and not others. 

Projective techniques also begin with the individual as the unit of analysis. 
Projective techniques involve the use of Incomplete or vaguely defined stimuli 
which the respondent is required to complete. It is assumed that in the process 

of interpreting and completing -the ambiguous stimulus material attributes are 

b 

revealed i which could * pot have been drawn out t$roug|fi the use of direct questions, 
because attributes are unconscious and cannot be v^b^ized. A projective tech- 
nique which may be potentially useful in eliciting important innovation 
attributes is the sentence completion test. This test has been used frequently in 

tt*rket-*ag research (Hughes, 1971). Completion techniques involve the presentation 
of an incomplete stimulus (e.g., sentence), with instructions that the respondent 
complete it any way he wishes. 

Brainstorming, which begins with the group as the unit of analysis, is a type 
of group discussion which attempts to Increase the involvement o.f individuals; the 
amount of interaction within the group; riduce inhibitions; and encourage creative 
thinking (Sampson, 1970). Synectics, also used with groups, is a technique for 
directing creative potential for the solution of problems (Gordon, 1961). Synec- 
tics is a Greek word which means the joining together of different and seemingly 
irrelevant items. This technique is based on the assumption that the psychological 
processes and mechanisms which take place in the creative process are usually un- 
conscious. The synectics group situation, which encourages each respondent to 
verbalize his thoughts about an object or a particular problem, can surface assump- 
tions so that they can be identified and analyzed (Sampson, 1970). 
Structured Techniques . Structured techniques usually make use of statistical 
methods for identification of important attributes. Three techniques which can 
be used for determining important innovation attributes include factor 
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analysis, cluster analysis and non-metric multidimensional scaling. 

Factor analysis involves reducing a large number of attitude dimensions to a 
few factors with minimal loss of information (for a technical discussion, see 
Nunnally, 1967). There are at least three ways that factor analysis may be use- 
ful in identifying important innovation attributes. They are: 

1. Separation and analysis of distinct dimensions that are latent in a 
larger set of innovation attributes 1 " 

, 2. Separation and analysis of distinctly different groups of stakeholders 
which exist in a larger population. 
3. Identification of certain likely attributes for subsequent multiattri- 
bute decision theory models (e.g., policy capturing) from a much larger 
set of potential attributes (Massey, 1964:p. 242). 
One of the major weaknesses of factor analysis is that the researcher is required 
to interpret and label the factors once the technique has reduced the data. 

Cluster analysis is a method for classifying objects. # This technique focuses 
on the objects by classifying together those stakeholders whose judgments or 
evaluations are similar along several dimensions (for a more complete discussion, 
see Green and Tull, 1970). As with any classification scheme, cluster analysis 
tries to maximize the homogeneity within clusters and the heterogeneity between 
the groups. The purpose of cluster analysis, then, is to describe or map objects 
in space according to respondents' attitudes, not to predict such a map (Hughes, 1971). 

Non-metric multidimensional scaling is similar to factor analysis in that 
hoth are data reducing techniques. These techniques .have t*o major advantages. 
First, in, one operation they are able to identify the dimensions and place them in 
n-dlmensional space. Second, they do not require data measured at the interval 
level (Hughes, 1971). An Important assumption of non-metric multidimensional 
scaling is that an n-dlmensional attribute space can be determined from unidimensional 
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data in which the distances represent the degree of similarity/dissimilarity among 
objects. Non-metric multidimensional scaling shares with factor analysis the dis- 
advantage that the- researcher must interpret and label the dimensions after mapping. 
Partially Structured Techniques . Partially structured techniques for identifying 
important dimensions of educational innovations involve systematic procedures for 
eliciting responses from subjects. The response set, however, is still determined 
by the respondent. As with other techniques, partially structured procedures can 
be subdivided by unit of analysis. 

Reason analysis, used with individuals, is a set of procedures used to con- 
struct causal explanations of decisions, actions or intentions (Lazarsfeld, 1935). 
In the area of innovation adoption, an Individual's judgment regarding the utility of 
a particular innovation may be subjected to reason analysis. When the purpose of an 
Investigation 1s to determine how a decision came about (i.e., what steps were taken, 
what influences were present at the time of the decision, what outcomes the actor 
expected, etc.) then reason analysis 1s appropriate. Reason analysis can assist 
the researcher to uncover the assumptions which an Individual decision maker 
brings tq bear in an action situation. 

Kelly's Repertory Grid, also used with individuals, is a modification of a 
testing procedure based on a highly developed and formal theoretical framework 
called personal construct theory (Kelly, 1955). Personal construct theory posits 
that man uses his cognitive 1 processes to predict and control his environment. As 
experience is gained, each individual learns to codify observations into a system 
of interrelated interpretations, thus developing his own set of personal constructs 
by which the individual structures his world and learns to anticipate future events. 
The psychology of personal constructs is concerned with ways to analyze and describe 
an individual's unique set of constructs as well as to discover how and why these 
sets develop and change (Bannister, 1962). 
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The Implications of this theory are that an individual responds to a situation * 
in terms of how he uniquely perceives the situatipn. In turn, an individual's 
interpretation of a situation is directly dependent upon his current repertoire of 
personal constructs. Therefore, the ability to understand and predict human 
behavior depends upon the extent* to which the relevant set of personal constructs 
can be surfaced (Bannister, 1962). 

For the purposes of surfacing relevant constructs, Kelly developed the Reper- 
tory Grid Testing Technique. A number of forms of the test have developed over 

the years, each of them tailored to specific purposes. The general method requires 
respondents to compare Innovations along dimensions that the respondent himself 

generates. Briefly, the technique is applied as follows: 

1. A list is drawn up of all the objects in a field under study. (e.g., a 
working^ universe of types of educational innovations. The length of the 
list is not critically important, but the optimum number is between 10- 
30 items or subsets. 

2. The items are then transferred onto a set of numbered cards, each of which 
carries the name of a single object (e.g., a specific innovation). 

3. A grid is prepared for recording the subject's responses. The columns of 
the grid are* headed by numbers which correspond to the numbered cards 
carrying the name of the object or subset. 

4. The interviews are carried out individually: a) the subject is shown the 
entire pack of cards and told to sort through them and remove any with 
which he is totally unfamiliar; b) three cards are selected in a pre=^ 
determined random order; and c) the respondent is asked "to think of a 
way in which any two of the three are similar to each other end different 
from the third" (Frost and Braine/ 1967:165). Thus, without being led, 
the respondent reveals a construct which he uses to categorize and 
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differentiate between objects (e.g., Innovations) 1n 

the field. , • 

5. Having identified a dimension by which the user simultaneously categorizes 
and differentiates between objects, the respondent is asked to sort through 

the cards and rate the remaining brands along this dimension. This rating 

/ 

may be in the form of a paired comparison, a ranking, or a rating 1 along a 
scale. \ 

\ 

6. Next the respondent is handed a different set 6f cards and the procedure 
is repeated (i.e., "to think of a way in which any two of the thrfe are 
similar to each other and different from the third"). In addition, the 
respondent is instructed not to repeat a response that was given previously, 
in order to generate a new construct. 

7. The process continues until the respondent can no longer think of new Con- 
structs. The average response is 18 constructs per interview (Frost and 
Bralne, 1967) . In most cases only 40 Interviews are necessary to identify 
all the relevant attributes in a domain of Interest. 

The results of the Grid Test may be analyzed in a number of different ways 
ranging from the very simple to the complex. Basically, the grid which results 
from this technique has the ordinary properties of a two dimensional matrix. A 
measure of association between any pair of rows may be computed by counting the 
number of columns on which they match or the number of rows on which they match* 
Non-parametric analysis of variance, factor analysis, and multidimensional scaling ' 
may be used further to reduce data. 

A partially structured technique used with groups for identifying attributes 
is a modification of the Delphi technique. The Delphi technique was originally 
developed for the purposes of improving the accuracy of forecasting technological 
change. The Delphi technique is an iterative procedure used to generate a 
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consensus among Individuals in s group with differing judgments about an object or 
^alternative. During the first round each individual is asked to give an opinion 
regarding a specific alternative. The results of the group's judgments are then 
tabulated and fed back to the respondents who are then asked to give another esti- 
mate. These iterations of judgments and outcome feedback are continued until the 
group consensus reaches a fairly narrow range. 

A modification of the Delphi technique ^an be used for determining important 
attributes of performance measures (Rortm^ andykulka, 1979). First it is necessary 
to develop a list of candidate attributes. These candidate attributes can be com- 
piled from a number of different sources (e.g., previous evaluation studies.). 
Next, a group of experts is asked to review the list of candidate attributes. Each 
candidate item is then evaluated by the respondents along several dimensions such as, 
essentiality (i.e., whether an item is likely to be included in a routine assess* 
ment of innovations. a smmnary of the first round responses are then 

graphically displayed, along with a reminder of the individual's own response* 7 to 
the original questionnaire and sent back to the respondent. During the second 
round the respondent is asked to indicate any changes in his assessments, based on 
the distribution of his colleague*' evaluations. This modified version of the 
Delphi technique has been used successfully in developing criteria for* the assess- 
ment of the quality of medical care (Roam and Hulka, 1979). 

Another relatively structured technique, but one that may be used with indi- 
viduals or groups, is the dialectical performance assessment technique (Dunn and 
Mitroff, 1980; Dunn, Mitroff, and Deutach, 1981). This technique permits researchers 
to analyze an individual stakeholder's unique set of attributes (assumptions) and 
to discover how and why these assumptions change as a consequence of group interac- 
tion and debate. While dialectical performance assessment involves explicit and 
systematic procedures tor eliciting judgments and underlying assumptions or 
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attributes, the response set is still determined by subjects who interect in the 
course of s dialectical debate (see Mason and Mitroff, 1981). The central differ- 
ence between this, technique end other procedures for eliciting interactive group 
judgment (e.g., Adelman, Stewart, and Hammond, 1975) is that dialectical perfor- 
mance assessment generates attributes from users themselves., While dialectical 
performance assessment and Kelly ' s • Repertory Grid can both elicit "base-line" 
measures of uniquely individual attributes, only the former does so as part of 
group transactions that may be monitored over time. The weakness of dialectical 
performance assessment is that it provides no systematic procedures for determining 
whether we have approximated the attrifeute-space of individuals or groups, whereas 
Kelly's Repertory Grid permits us to generate a working universe of attributes for 
individuals and groups. This property of Kelly's Repertory Grid clearly en „ 
advantage with extraordinary -implications for improving the validity of applications 
of multiattribute procedures to problems of adoption Of educational 

Innovations, and attempts to investigate the utilization of policy research in 
general. 

All standard multiattribute procedures of which we are eware fail to address 
satisfactorily what Hoffman (1960) calls the "paramorphic representation" problem. 
The term paramorphic — a combined form of "para" (falsely or inadequately) and 
"morphic" (structure or pattern) ~ calls attention to the fact that "the mathemat- 
ical descriptiun of Judgment is inevitably incomplete. . .and it'is not known how 
completely or how accurately the underlying process has been represented" (Hoffman, 
1960:125). Slovic and Lichtenstein (1971:675) observe that major regression end 
Bayeaian approaches to multiettribute decision theory — including Brunswik's Lens 
Model end probebilistic functionalism, social Judgment analysis, integration theory, 
end conjoint analysis — are alike in their incapacity to deel with the problem of 
paramorphic representation. Recent innovetions in multiettribute procedures 
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" (e.g., Saaty, 1980) also share this limitation; as do older and more established 
techniques such as multidiaensiqnal -scaling (see Kruskal and Wish, 1978). - Me 
believe that dialectical performance assessment and personal construct' theoty 
promise major improvement^ in the theory and applications of multiattribute decision 
theory, since both are based on a cons true tivist (rather than obj activist or rea- 
list) epistemology that demands the use of research procedures that elicit 
attributed from stakeholders or subjects -themselves. 

Explanation of Judgments 

All multiattribute procedures seek in some fashion to establish a plausible 
theoretical basis for explaining variations in judgments. The importance of theory, 
in this and other cases, is to predict the choice behayior of consumers and users 
on the basis of statements contained in the thiory (e.g., utility functions) that 
conform to experiitintally or statistically established covariations. As Slovic and 
Lichtenstein (1970:673) observe, multiattribute procedures are based on distinctive 
theoretical models that posit a variety of compositional rules by which subjects 
combine or integrate information to make judgments. These compositional rules fall 
into two broad classes, subjective and objective. 

Subjective and objective compositional rules assume diverse forms, some of 
which are illustrated in Table 3 below. Each of these compositional rules may be 
subjective (i.e., refer to the subject's judgment processes) or objective (i.e., 
refer to the subject's external environment). Conjoint analysis, building on 
(nonparametric) ANOVA designs, attempts to explain variations in judgments by 
positing additivWlnear compositional rules whereas functional measurement, 
building on (parametric) ANOVA designs, is a general heuristic that makes no prior 
assumptions about the form of compositional rules. Multiattribute utility analysis 
and social judgment analysis also provide additive linear compositional rules, 
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Table 3 

FORMS OF COMPOSITIONAL RULES 
WITH ILLUSTRATIVE EQUATIONS 



* orm Illustrative Equation 

LiJeAR (1) ADDITIVE U - b + £l b,x, 

° 1=1 1 1 

LINEAR ADDITIVE- U - b Q +21 b.x. + jr. 044X4X4 

INTERACTIVE ° 1-1 1 1 iTJ-l 1 J 1 J 

LINEAR (1) ADDITIVE WITH U - b_ + £ Ix? ' 

HIGHER ORDER FUNCTIONS 1-1 

MULTIPLICATIVE u - b^x^ 1 . 

1-1 



^The term linear refers to the form of the composition*! rule 
rather than to the form or power of terms contained in the composi- 
tional rule. 

SOURCE: Johnson end Huber (1977), Table II, p. 316. 
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although mulfciattribute utility analysis posits ^multiplicative rule as well 
(Keeney, 1975). Finally, the analytic hierarchy process is an elegant and unique 
mathematical theory based on a simple additive, linear compositional rule (see 
Saaty, 1980). h 

The relationship between objective and subjective compositional rules is 
perhaps moat explicitly addressed by social Judgment analysis. Building on the ^ 
psychology of Egon Brunewlk, including his Lens Model and its underlying theory 
<Df probabilistic f unctionalism (Brunswik, 1952; 1957), social Judgment analysis - 
uses multiple regression analysis to model the Judgment process. Each stimulus 
(alternative) is defined by feveral characteristics or attributes and measured 
quantitatively or with dummy variables. Respondents then assess each stimulus 
by providing an overall numerical evaluation. A regression equation estimates 
the contribution of each characteristic or attribute (independent variable) to 
the total variance in the overall numerical evaluation of each stimulus. Once the 
Judgment process has been modeled, according to a linear additive*composition rule, 
the beta weights fcan be interpreted as the relative importance a subject or Judge 
places on a particular characteristic or attribute. 

Social Judgment analysis is especially Important for our purposes because it 
seeks, along with Brunswik's probabilistic functionalism, to understand how indi- 
viduals learn and adapt to their environments. In probabilistic functionalism an 
individual's Judgment depends not only on his cognitive processes but also, and 
perhaps most decisively, on the tasks he performs (Brehemt 1980:14). The modified 
Lens Model of social Judgment analysis claims that the complexities of "wicked" or 
"squishy" epvironments prohibit 

objective specification of the causal relationship between 
that which is known and that which is to be inferred... 
Social Judgment theory provides a theoretical framework 
for understanding. . .disagreement (among policymakers) 
because it focuses primarily on the cognitive representa- 
tions of relations between the variables (Including their 
interdependences) in the environment (Adelman, Stewart, 
and Hammond, 1975:139-40). 

m£ ' ■ 2c; 
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Social judgment analysis seeks to represent individual's compositional rule 
by estimating four parameters: (1) the relative importance of attributes (proximal 
cues), expressed as weights; (2) the form (linear or nonlinear) of the functional 
relationship between attributes and judgments; (3) the organizing principle (addi- 
tive or non-additive) used to combine or integrate attributes into an overall 
judgment; and (4) the consistency (cognitive control) with which attributes are 
employed to make judgments in a dynamic external environment. Social judgment 
analysis externalizes properties of an individual's cognitive system, buf in'doing 
so avoids contrived and unrepresentative stimuli characteristic of factorial axper- 
iments. that systematically control levels of treatment conditions. 

Generally, social Judgment analysis and other multiat tribute procedures may 
employ three classes of tests to validate compositional rules (see Slovic and 
Lichtenstein v 1971:674): empirical correspondence between observed responses of 
subjects and those predicted on the basis of subjective compositional rules; 
empirical correspondence between observed responses and those predicted on th* 
basis of objective compositional rules; and coherence between responses predicted 
on the basis of subjective and objective compositional rules. Two major problems 
with these tests are, first, that they may be based on the selection of the less 
appropriate of two subjective or objective compositional rules, for example, a 
linear additive rule that does not capture interaction effects. This particular 1 
problem, perhaps best described as the isomorphic representation problem , has bee^ 
addressed by functional measurement, a multiatrtribute procedure that makes' no a 
priori assumptions about the form of compositional rules. Second, and more Serious, 
ia the paramorphic representation problem (Hoffman, 1960; 1968) already discussed 
in connection with the identification of attributes. | 

While problems of isomorphic representation can be minimized by various forms 
of triangulatiotl (Campbell, 1959; Webb et al. , 1966), including the three classes 
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°£^ests described above, problems of paramorphic representation involve substan- 
' tiye (rather than technical or analytic) questions about how~to uncover the 
attributes or assumptions of subjects without Imposing *n them the lAguage, con- 
structs, and procedures of the researcher. Indeed, only the dialectical performance 
assessment technique has attempted to develop procedures for identifying user- 
generated attributes. Thus, in contrast to other multiat tribute procedures, 
dialectic*! performance assessment is .assumption-focused , that is, directed towards 
the surfacing of attributes that are subjectively meaningful to stakeholders (sub- 
jects) themselves. All other procedures, since they have thus far avoided attempts 
to deal directly with subjective properties, are calibration-focused insofar as the 
only available tests to validate compositional rules are based on statistical gener- 
• alizations about the empirical correspondence between predictions and observed 
behavior and/pr the theoretical coherence of two or more compositional rules. 

Ultimately, these tests are based on a view of human judgment and decision 
that is not easily reconciled with recent developments in the logic of practical 
inference (Dunn, 1981). If all human Judgments are contextual — and if these 

judgments are rooted in cognitive processes that are indexed- to specific contexts 

then Compositional rules are indexical rather than lexfcal (see Knorr, 1981) . The 
lexical comifcpitional rules of standard multiattriSute decision theory — rules 
that are based on metaphors borrowed from algebra, one of the "ordinary" languages 
of social scientists — thus provide calibration-focused explanations t^at *re 
unlikely to overcome the problem of paramorphic representation. Needed are assump- 
tion-focused multiat tribute procedures.* 



^_ *Although the dialectical performance assessment technique- is assumption-focused 
if does not provide a plausible basis for explaining variations in Individual judg- 
ments;^^: is the hypothesis that conflict is a source of creative group judgment 
well-e«ablished. Thi* theoretical weakness is directly related to limitations 
(discussed above) in establishing plausible grounds for claiming that a stakeholder's 
attribute- or, assumption-space has been appropriately mapped. Any program of research 
that seeks to overcome theoretical weaknesses' rtusplbegin first by "filling" the attri- 
bute-sp.aces of stakeholders. . 4* 

♦ 



Unit of Analysis 

A third problem characteristic of attempts to apfcly multiattribute procedures 
is that of establishing a plausible basis for delineating the scope of involvement 
in multiattribute decisions. Thus far/ most multiattribute procedures have focused 
on segmented forms of involvement, that is, forms of involvement where a part or 
segment of some wider population of potentially Interested or influential actors 
participates in making judgments. For example, an individual policymaker,' whether 
real or hypothetical, is the focus. oF much of multiattribute .utility analysis (see 
Keeney , 1975) . Although extensions of MAXJT have sometimes focused on groups of 
stakeholders, the presence of intragroup conflicts and an attendant Incapacity to 
address such disagreements, leads advocates of this particular multiattribute pro- 
cedure to recommend segmental decisions by single top decision makers or narrowly 
based groups of experts: 

One function of an executive, boss, or decision maker is to 
resolve disagreements among subordinates. Be can, do this in 
various ways; by substituting his judgment for theirs, by ■ 
picking one of them as "right" and rejecting the others, or, 
in the weighted-averaging spirit of multiattribute utility , 
measurement, by assigning a weight to each of the disagreeing 
subordinates and then calculating weighted-average Importance 
weights (Edwards, Guttentag, and Snapper, 1975:159). . 

In contrast to segmented forms of Involvement that bypass or suppress con- 
flicts among decision maker* are comprehensive forms of . involvement. Here the 

idea of "stakeholders" is particulary important, since it calls attention to the 
fact that complex systems involve multiple decision makers who affect and are 
affected by policies developed for the system as a whole (see Mltroff , amhoff , and 
Kllman, 1979rDunn, Mltroff, and Deutsch, 1981). Conflicts among multiple stake- 
holders are not exhausted by formal-analytic limitations on group choice — for 
example, the formal-analytic impossibility of obtaining transitive group prefer- 
ences (Arrow, 1963) — but extend to substantial (rather than simply analytic) 
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disagreements about underlying causal assumptions, ethical principles, and goals of 
practice. By employing the concept of stakeholder as. a referent for all system- 
relevant actors DIAPAI accentuates comprehensive and representative involvement in 
multiattribute decisions. This orientation towards comprehensiveness enlarges the 
potential scope of attributes that complicate problems of judgment and Justify the 
use of multiattribute procedures in the first place. To be sure, single decision 

makers making independent choices bring multiple conflicting attributes to their 

t 

decisions, if only because their external environments are uncertain and fluid 
(see Brunswik, 1957; Adelman, Stewart, and Hammond, 1975). But once we begin to 
add system-relevant' actors (stakeholders) these uncertainties and fluidities 
accelerate exponentially, since stakeholders are judges and objects of Judgment "* 
at the same time. 

No standard multiattribute procedure of which we are aware has attempted to 
move beyond a focus on^segmental involvement. Conjoint analysis and functional 
measurement are typically employed under contrived end artificial laboratory condi- 
tions calling for segmental Involvement. While multiattribute utility analysis, 
social Judgement analysis, and the analytic hierarchy process acknowledge the 
importance of involving multiple decision makers, they do so unsystematically and 

• with little explicit concern for justifying selection processes. MADT, for example, 
speaks simply of "stakeholders" but without delineating their scope or characteris- 
tics, while social judgment analysis has tended to limit involvement to persons who 
appear most to Influence decisions, thus excluding many (who are affected by decisions. 
Although social judgment analysis seeks to maximize the variability of environments - 

• a procedure quite in keeping with Brunswik' s principle of "representative design" — 
it does not fully extsfed this principle to maximize the variability among individuals. 
Although Brunswik' s model can be "dialect icized," thus maximizing individual varia- 
bility (Mitroff, 1974), it may also be argued that probabilistic functionalism, 



ERIC 



' 2.L', 



along with other theories of social learning that rest on soma form of episttmAo- 
gical realism (Campbell, 1.974), are based on analogies to physical systems that ara 
Inapproprlata for tha social sclancas bacausa thay supprass variations in interpre- 
tlva structuras (Dunn, 1981). Finally, tha analytic hierarchy process, while . ^ 
potentially amanabla to segmental as well as comprehensive forma of involvement, 
simply acknowledges that one, several, or many "concerned persons" may be invplved 
(Saaty, 1980:8-9). . ~^ 

Tha dialectical performance assessment technique is the only existing multi- 
attribute procedure that alms at comprehensive involvement. Antecedents of D1APAT, 
while also aiming at comprehensive involvement, have in fact confined choice 
situations to segmental ones involving corporate and public managers and their 
staffs (Mason, 1969; Mltroff, Barabba and Kilmann, 1977; Mitroff and Eaahoff, 1979). 
While the involvement of all potentially relevant atakeholders max be agreed on 
grounds provided by classic theories of democratic participation (see Patonan, 
1970), it may also be debated on pragmatic grounds provided by systems theory 
(see Churchman, 1^968; 1979; Ackoff, 1974). If social py sterna are purposive entities 
composed of interdependent actors whose interpretive structures are often or always 
in conflict, one may well find that it is easier t/*solve many probl«na at once 
then to solve single problems sequentially. Since "problems" themselves are a 
function of conflicting interpretive structures — rather than "natural" entities 
standing alone in complex external environments — compreh^^/e involv«nent of 
stakeholders may reduce the probability of those practical errors and even disas- v 
tars that seem to be correlated with segmental decision making by "experts" (see 
Fischhoff, 1977). 

Nevertheless, claims about the desirability of comprehensive involvement in 
multiat tribute decision making are hypotheses, merely. No petsuaaive evidence has 
ten accumulated thus far to show that multlattribute procedures are superior to 
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ordinary bargaining and persuasion In reducing group conflict; nor has a good case 
been mad* that deliberate attempts to increase group conflict result in Improved 
decisions (see^Schwenk and Cosier, 1980). This Issue, as we shall see In the next 
section, depends not on the scope of Involvement, per se, but on the particular 
measurement process used to elicit individual and group Judgments. 

The Judgment Process 

A fourth probl^iound in attempts to apply multiattribute procedures is that 
of astablishing a plausible basis for representing the judgment process. In one 
way or another most advocates of multiattribute procedures claim that the Judgment < 
process used in conjunction with their distinctive approach not only yields adequate 
theoretical explanations of 4ariations in judgments/ they also claim or imply that 
a particular process facilitates improved decisions. Even those mult iat tribute 
procedures that aim primarily at the description and validation of psychological 

— that is. functional measurement and conjoint analysis — may also be 
regarded, at least lay implication, as efforts to delineate a repertoire or pool of 
lawful psychological regularities from which we might select those that are most 
conducive to improved judipiant. This, for example, research findings from con- 
joint analysis (Tversky and Kahneman, 1974) have been used to suggest ways*to 
identify universal biapes of evaluation researchers and to argue that greater 
critical awareness of established statistical principles, when juxtaposed to a 
variety of uncertainty-reducing yet error-prone heuristic procedures employed in 
everyday decisions, may reduce errors of Judgment (Perioff , Gadget, and Brock, 
198u:12, 18). . 

We may thus regard all multiattribute procedures as both descriptive end norma- 
tive-, that is, designed both to represent and Improve processes of Judgment. Existing 
differences among multiattribute procedures In this regard are, therefore, largely 
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matters of emphasis. Conjoint analysis and functional measurement ara primarily 
descriptive, dealing as thay do with tha discovary and validation of psychological 
laws, but implicitly normative. By contrast, multiattributa utility analysis, 
social Judgment analysis, and the analytic hierarchy process are primarily norma- 
tive — that is, designed to increase the if fectlveness and efficiency o£ choices — 
even though each is based on definite underlying theories whose origins lie" in 
statistical decision theory, applied mathematics, or learning theory. Finally, 
descriptive and normative dimensions may each be subdivided into two additional 
classes of aims that ara confirmatory (i.e., designed to validate established 
theories) or heuristic (i.e., designed to generate pw theories). These four alms 
of multiattributa procedures are described in Table 4. 

(Place Table 4 about hare) 

For present purposes the normative dimension of multiattributa procedures is 
most salient/ since our aim is to lmptove as well as describe the judgment process. 
This does not mean that we are not interested in describing tha Judgment process — 
■ince all normative procedures ntssarily involve description — but, only that our 
primary aim is facilitating improvements in the process of selecting Innovations for 
adoption, it should also be noted that rules for successful practice (e.g., recipes) 
do not necessarily require an understanding of causal links between rule and conse- 
quence. Here we join those who seek to devise new and better "decision aids" 
(Slovic and Lichtenstein, 1971:716-724), including those normative heuristics that 
are based on man-machine interfaces and computer technology (Edwards, 1962; Howell, 
1967; Schum, 1969; Dawes, 1971; Adelman, Stewart, and Hammond, 1975; Mason and 
Mltroff, 1981). 

Normative oriented multiattributa procedures are baaed on different claims 
about the Improvements we may expect if certain judgment processes are used to make 
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Table 4 

POUR AIMS OF MULTIATTRIBUTE PROCEDURES 

Primary Methodological Aim 
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Descriptive 



Normative 



Primary 

Theoretic 

Aim 



NOTES: 



Confirmatory 



Heuristic 



CONJOINT,. . 
ANALYSIS 


MULTIATTRIBUTE 
UTILITY ANALYSIS • 


FUNCTIONAL 
MEASUREMENT (3) 


SOCIAL JUDGMENT 

ANALYSIS W 
ANALYTIC HIERARCHY 

PROCESS^*' 

diapat(4) 



Descriptive confirmation focuses on psychological laws of everyday 
Information processing, for example, the' "lav of small numbers" 
(Tversky and Kahncman, 1971). \ 

(2Y 

Normative confirmation focuses on Bayes' theorem (Edwards and Phillips , 
1964) and axioms of choice (Keeney, 1975). 

(3>Efforts made to discover psychological laws governing the use of compo- 
sition rules to make judgments (Anderson, 1964; Meyer, Levin, and Louvieve. 
1981). 9 
(4>Efforts made to discover lawful regulations in judgment that reduce 
probabilities or absolut* levels of error, either statistical or concep- 
tual (we Hammond, ^965; Saaty, 1980; Dunn and Mitroff , I960). 
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decisions. Multiattribute utility analysis, for example, claims that stakeholders 
who follow a compositional judgmei|t process will make better decisions. Here the 
judgment process begins with multiattribute decisions that are subsequently composed 
(aggregated) in the form of a weighted average (Edwards, Guttentag, and Snapper, 

1975:156): ' 

- * 

U i-f w j u ij 

where U ± is th* aggregate utility for the i-th alternative, Wj is the normalized 
Importance weight of the j-th attribute, and u ±i is the scale value of the i-th 
alternative on the j-th attribute. The decision rule (0^ max) is compos it ioned 
insofar as it depends on prior judgments about the value or utility of alternatives. 
A similar compositional process underlies the analytic hierarchy process (Saaty, 198Q) 

By contrast, other multiattribute procedures claim that stakeholders (judges) 
who follow a decompositional judgment process will make better decisions. Here the 
Judgment process .beglns^with multiat tribute decisions that are subsequently decom- 
posed into Importance weights, for example, by using regression (beta) weights as 
indicsnts of 'Importance (Adelman, Stewart, and Hammond, 1975:14): 

Y. - BjGi + B 2 G 2 + ... + Bnft, + e 
where Y fl is the stakeholder •a (policymakers) judgment, ere attributes (cues, 
factors) upon which the judgment is based. Bj are beta weights that indicate the ^ 
relative Importance of eafch attribute (cue. factor) for the stakeholder's judgment, 
and e is a random error term. As an antecedent of DIAPAT strategic aasumptional 
analysis r (Mitroff and Emshoff , 1979) is also based partly on decompositional 
judgment processes. 

Compositional and decompositional variants of multiattribute decision theory 
make normative claims about the beneficed effects that follow when their respec- 
tive Judgment processes are used by decision makers. Generally, compositional 
Judgment processes are believed to reduce bias, while decompositional proteases 
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are believed to reduce the unreliability {inconsistency) of judgments (Slovic end 
Lichtenstein, 1971:721-722). Thus, Edwards ,\Gutten teg, and Snapper v ( 1975: 158-159) , 
in claiming that MAUT reduces interpersonal disagreements, offer the following 
explanation: 

When making wholistic evaluations, those with strong 
points of view tend to 'concentrate on those aspects 
of the entities (alternatives} being evaluated that 
most strongly engage their biases. But the multi- 
attribute procedure does not permit this; it separates 
judgment of the imp or tan am of a dimension from judg- 
ment of where a particular entity falls on that 
dimension... Agrjement (qn some non-controversial 
dimensions) tends to water down disagreement: on con- 
troversial dimensions in a multiattribute utility . 
procedure. 

By contrast, social judgment analysis claims that stakeholders who employ its - 

i 

characteristic judgment processes will experience reduced levels of conflicjt, 
distrust, and hostility, but for different reasons. Social Judgment analysis 
asserts thafc "policy quarrels can be described in terms of the cpgnitiye* para- 
meters (weights, function forms, organizational principles, and consistency) of 
the participants (Adelman, Stewart, and Hammond, 1975:140). 

The distinction between compositional and decompositions! judgment processes 
is related to another important difference among mult 1st tribute procedures. Some 
multiattribute procedures are compensatory; meaning that thaoges in the value >. 
of one -attribute can be off sat by- or traded for ^commensurate changes in soma other 
attribute. Other multiattribute procedures are noncompensatory , since changes in 
the values of one attribute cannot be offset or traded against an increase (or 
decrease) in the values of other attributes (see Green and Wind, 1973:38-46). 
Johnson and Huber (1977:316) make this same distinction in terms of "scoring" and 
"screening" models. 

All multiattribute procedures except D LAP AT are compensatory insofar as they 
represent judgment processes in terms of wholistic assessments of the value or 
utility of alternatives < While compensatory procedures are based on different 
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compositional rules (see Table 3) f each assumes that attributes may be. traded off 
in some fashion. Noncompensatory procedures, by contrast, assume three principal 
forms: dominance, conjunction-disjunction, and lexicographic ordering (Green and 
Wind, 1973:40-42). Strictly speaking, compositional procedures cannot also be 
noncompensatory since they posit judgment processes that involve holistic assess- 
ments of different kinds. Yet it is clear that judgment processes may also be 
represented as "phases" where different processes of judgment are brought to bear 
in making choices over time. Thus, for example, DIAPAT begins with a decomposi- 
tional-compensatory process where stakeholders make overall judgments about 
alternatives that are subsequently decomposed into asswptions (attributes of 
alternatives that are scaled by their importance and certainty as elements of 

an overall argument (see Dunn, Mitroff, and Deutsch, 1981). Once these attributes 

> 

have been plotted on a grid, subsequent judgments are noncompensatory insofar as 
a disjunctive decision rule serves to accept assumptions that fall ipto the quad- 
rant that involves the most important and least certain assumptions. Thts procedure 
sets the stage for a dialectical debate (Mitroff and fits toff, 1979). The general 
form of the disjunctive judgment process may be represented as: 
Given nonempty subsets of X, X*, xj, x£, 

* " &Xt x 2 » • • • • *j • • • • • *m) valued only if * 
Xj fcXj for some j (j - 1, 2, m). 

In Table 5 properties of DIAPAT and other multiattribute procedures are compared 
and contrasted. f 

(Place Table 5 about here) 
A final distinguishing characteristic of multiattribute procedures — and one 
which again differentiates DIAPAT from other procedures — is the mode of involve- 
ment in multiattribute decisions. Generally, involvement may take one of three 
forms: active, interactive , and transactive * Active involvement of judges is a 
characteristic of conjoint analysis and functional measurement, where individual 
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Tabic 5 

y 

PROPERTIES OF MULTIATTRIBUTE 
JUDGMENT PROCESSES 



Compositional 



Decompositidnal 



/ 


MULTIATTRIBUTE 


SOCIAL JUDGMENT 




UTILITY ANALTSIS 


ANALYSIS CONJOINT 


Compensatory 








ANALYTIC HIERARCHY 


FUNCTIONAL MEASUREMENT 




PROCESS . 




Noncompensatory 


MULTIATTRIBUTE 
UTILITY ANALYSIS (1 > 


DIALECTICAL PERFORMANCE 
ASSESSMENT^) 



NOTES: (1 >When rasults of MAUT ara "screened" by 
top dacision makars or axparts va as suae 
tfhat it is a phasa nodal raquiring tha usa 
^ of soma noncompensatory (domlnanca , con June - 
tive-diejunctive, lexicographic) dacision rule, 

(2 >DIAPAT ia explicitly designed in several 
phases, the second of which involves the use 
of a disjunctive and noncompensatory declelon 
rule* 
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stakeholders (Judges, consumers) cake elective role In making choices but do not 
directly share their judgments with other stakeholders. Interactive involvement 
is a characteristic of multiattribute utility analysis* social Judgment analysis, 
and the analytic hierarchy process. Each of these three procedures involves dif- 
ferent forms for "externalizing" — that is, making public or objectifying — 
Judgments. The most general of these externalizing processes is that of multi- 
attribute utility analysis, where "discussion, persuasion, and information 
exchange should be used in an attempt to reduce the disagreements as much as 
possible" (Edwards, Guttentag, aftd Snapper, 1975:159)". Slightly more focused are 
procedures recommended by Saaty (1980:9), nhe** each member of a group with con- 
flicting interests may be instructed to 

develop the outcome using his Judgments and assuming 
Judgments for the other parties, note the outcome, 
and compare it (perhaps with the aid of a computer) 

j with what the others arrive at. The process reveals 

I what outcome each party is exerting pressure to 
achieve. The crucial upshot of this is to induce 
cooperation. 

Tha most specific and operational procedures for structuring Interactive 
involvement developed thus far and those of social judgment analysis. Here the 
use of computer graphics technology provides "(1) lnaediate statistical analysis 
of the Judgment process, in terms of weights, function forms, and consistency 
(this statistical analysis is called 'policy-capturing 1 ), and (2) lxnediate 
pictorial description of these parameters (this is called 'externalization 9 ). 
In short, they permit an accurate description of the cognitive process that is 
the basis for disagreement" (Adelman, Stewart, and Hanaond, 1975:141). 

Transactive involvement of Judges is characteristic of DIAPAT, but not of 
other multiattribute procedures. While transactive involvement is also interac- 
tive, since stakeholders do externalize and share publicly their Judgments, 
DIAPAt also involves an explicit process for transacting the meaning of such 
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judgments. The term "transaction 91 refers to processes of examining the assumptions 
(attributes) brought to bear in affirming and contesting the plausibility of judg- 
ments, all of which are viewed as putative knowledge claims. Dunn (1981) has 
described a transactional model of knowledge (information) utilization based on 
a structural model of argument developed by Toulmin (see Toulmin, 1959; Toulmin, 
Rieke, and Janik, 1979) and extended to policy analysis and planning by Meson and 
Mitroff a 981). 

One of the central alms of this transactional model is to evoke £h4> taken- 
for-gr anted assumptions r that are tacitly employed to assess atatiatic^Mand 
causal inferences that are externalized by social judgment analysis in the form 
of computer graphics (e.g., the "cognograph"). Transactional involvement, while 
it involves conflict, bargaining, and Negotiation, is not confined simply to th# 
consideration of maximally different alternatives (compare Schwenk and Cosier, 
1980), but extends to the reflective examination of the underlying cognitive • 
grounds of disagreements among atakaholder*v_^£trfff (1981) claims that the trans- 
actional model is particularly well-suited to research into competing standards 

of assessment used in real -world settings to assess theoretical and practical 

- / 

claims alike, since it "provides a visual schema for mapping arguments, compels 
a reflective and critical posture towards (hidden) presuppositions of knowledge ^ 
claims, yields a classification of truth, relevance, and cogency tests, and 
permits a provisional listing of threats to usable knowledge." 

In surveying alternative multiattribute procedures this review has moved 
sequentially through four important issues that involve the identification of 
attributes, the explanation of judgments, the choice of units of analysis, and 
the representation of: the judgment process. We have deliberately avoided tech- 
nical issues, not only because such questions are adequately treated in available 
literature but because we wished to identify salient methodological properties 
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of procedures. As this r«v1ew makes clear, standard procedures for class- 
ifying multl attribute procedures are often based on surface technical 
distinctions that obscure rather than clarify their strengths, limitations, 
and potential applications. Moreover, 1t seems clear that the kinds of 
procedures that may alleviate the problems we have raised are likely to 
require a significant departure from past traditions or, to use a now. 
hackneyed phrase, a movement towards "new paradigms" of multl attribute- 
decision theory. What we have called the dialectical performance assess- 
merit technique represents 1n many respects an attempt to shape a new 
paradigm. This 1s evident when we compare and contrast, 1n summary form, 
DIAPAT and other multlattrlbute procedures (Table 5). 

CONCLUSIONS 

This working paper began by arguing that judgments regarding the 
utility of educational Innov^ions 1s best viewed as a multlattrlbute pro- 
blem. In the context of school Improvement, a multlattrlbute problem 1s 
one 1n which evaluation of an educational Innovation depends- on the sub- 
jectlve standards of assessment (attributes, assumpt1ons\employed by 
stakeholders who attribute different properties to education a^d systems 
to measure educational outcomes. We then compared and contrasted multi- 
attribute procedures according to four key methodological dimensions, 
Including the grounds for Identifying attributes, the forms 1n which they 
are externalized and explained, the scope of representation 1n decisions, 
and the mode of Involvement 1n such decisions. 

We found that the majority of multlattrlbute procedures differ sub- 
stantially from DIAPAT, although there 1s at present no*conv1ndng evidence 
that DIAPAT^r Its antecedents (strategic assumptlonal analysis) are superior 



Table 5 

COMPARISON OP MULT IATTR I BUTE PROCEDURES 



Property 



Procedure 


Ground of Attributes 


Focus of Explanation 


Sdope of Involvement 


Mode of Judgment 


MULTIATTRIBUTE , 
UTILITY ANALYSIS 


Researcher -Imposed 


Callbratlon-Focuied 


Segmental 


Active 'or 
1 Interactive 


CONJOINT ANALYSIS 


Researcher -Imposed 


Calibration-Focused 


/ Rental 


Active 


FUNCTIONAL 
MEASUREMENT 


Researcher-Imposed 


Calibration-Focused 


Segmental 


Active 


SOCIAL JUDGMENT ~ 
ANALYSIS 


Researcher-Imposed 


Calibration-Focused 


Segmental 


Active or 
Interactive 


ANALYTIC HIERARCHY 
PROCESS 


Researcher-Imposed 


Cal Ibra t Ion-Focused 


Segmental 


Active or 
Interactive 


DIALECTICAL PERFOR- 
MANCE ASSESSMENT 


User-Generated 


Assumption-Focused 


Comprehensive 


Transactive 



to other multi attribute procedures in improving complex decisions. Finally, 
we observed that methodoloaicaj properties of the majority of multi attribute 
procedures are inconsistent with requirements -for alleviating ill-structured 
problems, generally and in the specific context of measuring judgments of 
the utility of an innovation. Yet this claim t$ based on essentially 
intuitive and logical grounds, a weakness that may be remedied through 
directed research. 
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ABSTRACT 



In an attempt to untangle the field of knowledge use, 
this paper develops an extended typology designed 
to capture the most basic properties of present-day 
theories of knowledge use. Rather than siding with 
any particular theory, or an ideal-type of which 1t 
is an illustration, each theory is taken as a "datum". 
At a metatheoretical level , propositions and attendant 
corollaries are generated fro making statements about 
ideal -typical theories as a whole. 



USABLE KNOWLEQGE: A METATHEORY OF 
POLICY RESEARCH IN THE SOCIAL SCIENCES* 

William N. Dunn 

University Program for the Study of Knowledge Use 
University of Pittsburgh 

/This paper outlines a metatheory of policy research in the 
social sciences, that is, an organized set of propositions about f 
] 
existing theories of usable knowledge. By -outlining the main con- 
tours of a metatheory of policy research I seek, first, to counter 
the regrettable present trend towards conceptual entanglement that, 
now characterizes much theoretical wgrk on the uses of social re- 
search in public policymaking. Second, by engaging in a process 
^ of reflective theory-building I hope to uncover and raise to a 

level of explicit consciousness many unexamined assumptions and 
hidden standards of assessment that now impair or corrupt our under- 

t: 

standing of the role of policy -research in social problem solving. 
Finally, I want to supply a metatheoretical structure that promotes 

*Paper prepared for a conference on Non-Traditional Approaches 
to Policy Research, School of Business Administration, University 
of Southern California, November 11-13, 1981. Portions of this 
paper were prepared under Grant Number NIE-G-8 1-0019 from the 
National Institute of Education, Research and Educational Practices 
Program, U.S^ Department of Education, Points of view or opinions 
are those of the author and do not necessarily represent the official 
position or policies of the U.S. Office of Education or its com- 
ponent units. I should like to acknowledge the contributions of 
Marija Dunn to the design of the last section of this paper.' 
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at least marginal improvements in our capacity to understand and 
planfully * shape the production of usable knowledge. 

My major substantive c]/aim is that the understanding and con- 
duct of policy research may be improved by adopting propositions 
and principles of "critical constructivism," a metatheory of know- 
ledge use with eclectic multidisciplinary origins. In making this 
claim I should state at the outset that I am not interested in siding 
with any partfbular theory; nor do I propose to provide a quick 
"paradigmatic fix" that will suddenly heal some ailing worldview. 
My aim, rather, is to create a metatheoretical structure that per- 
mits and even compels a more reflective posture towards the role , 
and limitations of policy research in the social , sciences* 

Arguments offered in support of my clailn are organized in four 
sections. , In the first section I compare and contrast alternative 
definitions of knowledge use, argiuing that a divisipn of phenomena- 
to-be-explained (explananda) into "subjective" versus "objective" 
properties represents a suitable starting point for sorting existing 
theoretical perspectives. The structure of the second section is 
similar to the first, except that here I compare and contrast altern- 
ative explanations of knowledge use, arguing that a division of 
explanations (explanantes) into "imposed" versus "generated" 
phenomena provides an appropriate additional basis for distinguishing 
theoretical perspectives . The third section creates, on the basis 
oi; the inter-section* 6 ^: explananda and explanantes of knowledge use 
presented in the first two sections, a four-fold typology of 
theoretical perspectives.^ These perspectives are 4*hen further dif- 
ferentiated according to characteristics of three media (knowledge, 



-3- 



users, and social systems) through which imposed versus generated 
explanations of knowledge use operate. The product is an expanded 
typology that captures many V the most important theoretical 
claims and assumptions abotft the role of policy research in pro- 
ducing usable knowledge. Finally, the fourth section outlines I 
propositions and corollaries of "critical constructivism." The 
argument is that this particular metatheoretical posture not only 
helps uncover and explore complementary; features of existing 
perspectives; it also provides norms for developing new and' more 
appropriate theories of policy research that enlarge the domain of 



usable knowledge— that is, knowledge that enhances our collective 

\ 



learning capacities. 

\ 

KNOWLEDGE USE: THE CRITERION PROBLEM \ 

One of the most basic questions about any field of inquiry \ 
surrounds the criteria i^sed to define its subject matter. At 
present it is difficult to bound the field of knowledge use in 
meaningful arid persuasive ways* chiefly because the field is highly 
fragmented and conceptually "soggy" (Weiss, 1977:11). As Ganz 
(1981:186) observes, there has been a "continued adherence to an 
•iron triangle' between creation, diffusion, and utilization with- 
out reference to the complex interactions among these activities, 
institutional arrangements, and social change... 1 " Researchers r 
have tended "to think of themselves as working—or at least con- 
centrating — on knowledge creation, knowledge, diffusion, or know- 
ledge utilization/?""*"^ a result, these three subfields in the 

I 
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study of knowledge have evolved relatively independently of each 
other" (Rich, 1981a:7). Thus far, there is no plausible theoretical 
foundation that links these 'three subfields, while the "tangled 
literature" (Nelson and Winter, 1977) of the field as a whole 
reflects the absence of a minimal set df criteria for defining 
knowledge use as a dependent variable or phenomenon-to-be-explained 
(Zaltman, 1980; Dunn and Holzner, 1981). 

JJnder such conditions several alternatives ^re available to 
those who wish to impart more structure and coherence to the 
field. Along with LgLndblom and Cohen (1979), Wil<|avsky (1979), 
and Knott and W^ldavsky (1980) \ we . can elevate our present diffi- 
culties to the status of an ontological principle ,\ announcing 
that the "nature" of the policy-making process or of knowledge 
itself imposes essentially unalterable constraints ,on the creation 
of usable knowledge. Here we run the risk of attributing proper- 
ties of "irrationality" to social systems when, in fact, (ir)- 
rationality is a property of the constructs we ourselves offer' to 
explain them. Alternatively, we may acknowledge the complexity 
of knowledge use in contexts of practice, arguing that contextually 
determined definitions of knowledge and of use are unique indexical 
expressions (Knorr, 1981). Here we risk the abandonment of know- 
ledge use as a "pseudo-problem" when, instead, it is an authentic 
problem whose generalizable dimensions we are simply unable to 
penetrate with the tools at fiand. A third alternative is to 
develop a loosely organized multidisciplinary strategy, appro- 
priating whatever insights may be available in existing social 

science disciplines (sociology, political science, economics, 

, i ■ 
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psychology, anthropology) and their many subfields {sociology of 
knowledge, economics of information, cognitive psychology, applied 
anthropology) and hybrids (planned, social change, communication! ' 
of innovations, information sciences, artificial intelligence). 
Here the major risk is an unreflective brand of eclecticism and 
a diffuse labelling process that may further entangle or even knot 
orientations in the field. 

Another alternative is to begin with the field as it is, 
taking theoretical diversity and conflict as a datum or problem- 
atic situation which we might organize in more plausible and pro- 
ductive ways. Here we aim not at "solving" the problem — for 
example, by employing a favored discipline-based model or hybrid 
which yields explanations that shun the very complexity we wish 
to make intelligible — but at "structuring" the problem as a 
necessary prelude to eventual solutions. To .do so, however, 
requires that we systematically uncover competing theoretical 
perspectives of knowledge use and their underlying assumptions. 1 
In this way we may discover complementarities as well as conflicts 
among different perspectives, avoiding any mistaken impression 
■ that a single theory adequately represents the role of policy re- 
search in producing usable knowledge. 

The discussion that follows' attempts to Classify or sort 

theories of knowledge use according to answers provided to two 

2 V 
questions: (1) How is knowledge use defined ? (2) How is knowledge 

use explained ? The first question leads us towards alternative 

assumptions about knowledge use as a phenomenon- to-be-explained 
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(explanandum) , while the second yields assumptions about classes 
of phenomena which explain knowledge use (explanantes) . By pur- 

" i 

suing both questions at once we may avoid the kinds of difficul- 
ties that. arise when twd or more perspectives based on a common 
definition of knowledge use — and, thus, may be viewed as the- 
oretical cohorts — also propose fundamentally different explanations. 
The same situation arises in reverse when two or more perspectives 
are based on a similar explanation of entirely different phenomena. 
Accordingly, the plethora of conflicting research findings and 
conclusions that characterizes the field is not due so much to an 
absence . of definitions of "utilization," or even of multiple out- 
come measures (Larsen, 1981:151), but to the presence of conflicting 
and/or mixed definitions and explanations of knowledge use, 'how- 
ever implicit or ambiguous these might be'. 

Existing definitions of knowledge use seem to share at l^east 
one property in common: Knowledge use is a cognitive relation 
among two or more purposively behaving actors. By specifying that 
knowledge use is a "cognitive" relation we mean that the accompan- 
iment of one actor's behavior by that of another is somehow 
affected by the act or faculty of knowing. Indeed, it is this 
cognitive stipulation, and perhaps this alone, that differentiates 
knowledge use from disciplines or fields that investigate other 
types of relations based, for example, on powe»r, status, or ex- 
change . ^ « 

By ^stipulating that the behavior in question is purposive we 
call attention to the teleological character 6f knowledge use. The 

2 H J 
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striking common feature of otherwise conflicting definitions of 
knowledge use is their dependence ori assumptions about the motives,- 
intentions, or purposes of "producers" and "users." In one of the 
broadest classifications available Machlup (1962, 1980), for 
example, extends Scheler's three-fold classification of knowledge 
(intellectual, instrumental, Spiritual) by providing a teleological 
contrast among intellectual, practical, spiritual, pastime, and 
unwanted knowledge. Classifications of use--for example, "concep- 
tual," "instrumental," and "symbolic" uses of knowledge (Rich, 
1975; Capiat, 1979; Knorr, 1977)— follow a similar pattern of tele- 
ological reasoning. Indeed, it is difficult to formulate a satis- 
factory definition of knowledge or of use that does not hinge on 
the purposes of "knowers" or "users" (see Churchman^ 1971). 

Agreement on this abstract definition of knowledge use begins 
to dissipate when we consider specific features of existing per- 
spectives. While knowledge use may be a cognitive relation among 
purposively behaving actors, this cognitive relation may be de- 
fined in two fundamentally different ways: one type of definition 
emphasizes objective , observable, and overt cognitive relations 
among purposively behaving actors while the other stresses sub- 
3_£^iy e ' n on-observable, and covert cognitive relations. 4 The 
differ ence between these definitions, as Wallace (1969:6-7) observes 
of social relations in general, 

corresponds to j the difference between the Newtonian 
and Weberian definitions of "action." Newton's defini- 
tion of mechanical action was entirely in terms of 
external observables . . . Weber took exactly the 
opposite view by asserting that bodies 1 social action 
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is so far distinguishable from mechanical action as 
to k*p definable only in terms of their internal 
states. 

Although there may be and often is considerable overlap be- -~ 

tween these two types of definitions in particular theories of 

I f . 

knowledge use, it remains possible to characterize theories in ! 
terms of their emphasis on one type of cognitive relation rather 
than the other. Thus, for example, while Machlup includes in his 
definition of knowledge use both the "process of informing" (an 
objective cognitive relation) and the "message or knowledge con- 
veyed" by this process (a subjective cognitive relation) , he 
nevertheless defines the use of . knowledge as a subjective cogni- 
tive relation: 5 , 

If one wants to b£ consistent in keeping separate the 
process of informing and the message or knowledge Con- 
veyed by that process, one would be well advised to 
avoid altogether speaking of "uses" of information, 
except if one wishes to refer to choices among 
alternative modes of information... use of a mode of 
information should not be confused with the use of 
the message or knowledge conveyed (Machlup, 1979:65). 

The definition' of knowledge use as a subjective cognitive 
relation is evident in many other perspectives. House (1981:12-13), 
for example, recommends that investigations of knowledge use in 
.education be "directed at the different •meanings 1 produced by the 
change efforts rather than at the change itself," citing an example 
where an unsuccessful attempt to introduce PPBS was accompanied 
by extensive culture conflict (see Wolcott, 1979). Similarly, 
Aaron (1978:159) stresses that the use of research and experiment- 
ation on poverty, while it is seemingly unconnected to specific c 
overt changes in the behavior of policymakers, nevertheless- 
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"corrodes the kind of simple faiths drfwKich political movements < 
are built." In the same vein, Patton\s vgjb^T)bn utilization- 
focused evaluation defines knowledge us^hi term^_of "more pene- 
trating perspectives, increased capabilities^ and greater commit- 
ments to action" (Patton, 1$78:290). Caplan, while acknowledging 
the importance of objective cognitive relations associated with 
"instrumental" use, stresses the importance of subjective relations' 
found to characterize "conceptual" use among high-level policy- 
markers confronted by macro-level problems of high complexity 
(Caplan, 1979:465) . * " 

In contrast to an emphasis on subjective cognitive relations 
we find those who stress objective, observable, and overt cognitive 
relations. Here there is no apparent reference to underlying pur- 
poses, motivations, or intentions. Thus, for example, knowledge 
use has been viewed as an overt process of adopting and implementing 
research-based innovations (National Institute of Education, 1978), 
or as a change in the structure and functioning of a social sys- 
tem (Rogers, 1973). These definitions of knowledge use, which 
emphasize directly observable consequences of acting on kn^M^dge — 
that is, an objective cognitive relation — assume that use occurs 
when an entire set of recommendations is implemented in a form sug- 
gested by researchers (Larsen, 1981:150). 

Perhaps the clearest example of a definition of knowledge use 
in terms of objective cognitive relations is provided in a study 
Qf the use of evaluation research in criminal justice (Larson and 
Berliner, 1979). Noting that "evaluation is a process that pro- 

S 
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duces information to assist in the allocation of resources," the 
authors go on to define use in terms of the "decision-consequential 
impact" of research on decisions involving "an irrevocable allo- 
cation of resources" (Larson and Berliner, 1979:2).' Quoting 
Howard (1966) , the authors contend that "a decisidn is not a 
•decision to make a decision,' but rather the concrete action 
Applied by the decision" (Larson and Berliner, 1979:23n) , 6 

The distinction between subjective and objective cognitive 
relations, while it provides a suitable basis* for comparing and 
contrasting the explananda of theories of knowledge use, fails to 
acknowledge several additional distinctions . For example, altern- 
ative distinctions involving macro and micro ^levels of use have 
been employed by Caplan (1979), while more differentiated class- 
ifications of types of use are available in the literature (e.g., 
Larsen et al . , 1976; Weiss, 1976; Hall and Loucks, 1977; Dunn, 
1980; Knott and Wildavsky, 1980; Zaltman, 1980; Rich and Goldsmith, 
1981) . The most appropriate position towards these alternative 
distinctions is simply to argue that the subjective-objective dis- 
tinction captures many of the most important definitions current 
in the field. Indeed, the subjective-objective distinction 
uncovers and clarifies behavior now treated in these classifications 
as if it shares some single underlying property. For example, 
Knott and Wildavsky (1980:546) propose a seven-fold typology of 
"standards of knowledge utilization," some of which (e.g., cogni- 
tion and reference) conform to what we have- defined as a subject- 
ive cognitive relation, while others (e.g., implementation ^nd 
impackj^are instances of objective cognitive relations^ 
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Finally, the use of the subjective-objective distinction to 
differentiate explananda of theoretical perspectives omits dis- 
tinctions based on ordinary and scientific knowledge (Lindblom 
and Cohen, 1979) frames of Reference* (Holzner and Marx, 1979), 
truth tests and utility tests (Weiss and Bucuvalas, 1980), and 
policy sectors, arenas, and institutiortal contexts (Rich, 1979; 
J. Weiss, 1979). This omission is deliberate, since it is these 
and similar distinctions that permit us to differentiate the 
explanantes of theories of knowledge use. 

KNOWLEDGE USE: THE PROBLEM OF EXPLANATION 

Apart from their different emphases on subjective ax^i objec 
ive cognitive relations, available theories also offer dif£ 
classes of phenomena to explain knowledge use. Theories which 
adopt a common definition of knowledge use as a subjective cogni- 
tive, relation (e.g. , "conceptual" use) frequently advance differ- 
ent explanations of this same phenomenon—for example, explanations 
derived from knowledge-specific, policymaker constraint, or two- 
communities theories of knowledge use (see Caplan, Morrison, and 
Stambaugh, 1975). A similar situation sometimes exists in reverse, 
since authors who share the same explanatory schema — for example, 
an interactive, pluralistic, and incremental theory of policy- 
making — may hold fundamentally different definitions of knowledge 
use. Thus, while Lindblom and Cohen (1979:12) affirm that^ "know- 
ledge is knowledge to anyone who takes it as a basis for action," 
Wil£avsky ^(1979:27-28) and Knott and Wildavsky (1980) reserve the 
term "knowledge" for t 
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.. .theories relating policy variables to effects where 
' the principles are confirmed by the empirical test of 
repeated decisions;.. When policy makers are certain 
'that manipulating these variables will produce the 
expected effects — that is, if "x" is done, "y" will 
follow with a known probability — then they have know- 
ledge (547, 546. Emphasis supplied,). * 

Whatever elke one may say about interactionist explanations of know- 

ledge use, in this case their authors hardly share a common or 

cohesive theory — they are attempting to explain entirely different 

' phenomena • 

The principal phenomena offered to explain knowledge use may 
be grouped into jtwo main classes: conditions that are imposed on 
users by the given nature of knowledge, of users themselVes, or 
of the social systems' in which they interact; and conditions that 
are generated by users as they create knowledge, their own stand- 
ards for assessing knowledge , or the social systems of which they. 
^ are members. The first type of explanation implies that knowledge 
use is "natural," "determined, " or "imperative," while the second 
suggests that knowledge use is "artificial," "self-determining," 
or "constructed." 7 / 

Theories of knowledge use which provide explanations stressing 
imposed phenomena are sometimes offered by those concerned with the 
nature of social science knowledge and its affects oh research utili- 
zation. For example, Bernstein and Freeman (1975) , observing that 
"there have been few cases of actual effective utilization of 
evaluation research for expected purposes" (5) , offer an imposed 
explanation based on assumed characteristics of "quality" research 
and the attendant failure of federal agencies to ensure 
production. "Quality" research is defined as research that 
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conforms to standards of experimental and quasi-experimental design 
(100-101) , while\he failure to enforce such standards is described 
in the following terms: 

•••there is neither a federal evaluation policy nor a 
set of requirements and guidelines regarding what con- 
stitutes an appropriate evaluation. This lack has 
severe consequences. It results in a failure to enforce 
any standardized set of evaluation requirements, even 
when present in legislation (Bernstein and Freeman, 
1975:6). 

Whereas this and related perspectives (e.g., Yin et al., 
1976) seek to explain decision processes and other objective cog- 
nitive relations primarily in terms of imposed knowledge-specific 
characteristics, others stress imposed explanations based on assump- 
tions abcut the nature of users or their environing social systems. 
For example, Mitroff and Mitroff (1979) employ Jungian personality 
theory, which assumes given underlying personality structures or 
traits, to explain patterns of interpersonal communication among 
•knowledge users and producers. Whereas this imposed explanation 
focuses on user-specific characteristics, others stress phenomena 
that operate through social systems. For example, the weight of 
arguments offered in support of theories of a "post- industrial" 
society (Bell, 1973) is borne by explanations that stress condi- 
tions which are imposed by social system-specific characteristics — 
particularly, the proposition that the growing interdependencies , 
complexity, and pace of change of contemporary society make existing 
knowledge obsolete, thus increasing the demand for new forms of 
policy-relevant knowledge (see Straussman, 1978). 
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Theories of knowledge use are also based on explanations that 
emphasize phenomena which are generated by users as they create 
knowledge, their, own standards for assessing knowledge, or the social 
systems of which they are members. For example, Carol Weiss 
outlines three classes of generated conditions which parallel dis- 
tinctions among knowledge-specific, user-specific, and social 
system-specific characteristics: knowledge-driven, decision- 
driven, and interactive models of knowledge use (C. Weiss, 1977: 
11-14). In her own work Weiss has offered explanations based 
primarily on phenomena generated by users — specifically, the 
truth tests and utility tests that constitute the frames of refer- 
ence of decision makers (Weiss and Bucuvalas, 1980). Whereas 
Lindblom and Cohen (1979) are concerned with knowledge-specific 
characteristics in general—that is, with relations between "or- 
dinary" and "scientific" knowledge— Weiss and Bucuvalas specific- 
ally address knowledge* as a product of user-specific frames of 
reference. That they are not principally interested in social 
system- specific characteristics is evident in their description 
of their enterprise as a classical sociology of knowledge "turned 
upside down" (see Holzner, 1978:8): 



Whereas the classical sociology of knowledge was 
concerned with the social bases of intellectual 
productions, a sociology of knowledge application 
would be concerned with the social bases of intel- 
lectual productions, a sociology of knowledge 
application would be concerned with the social 
consequences of knowledge. . .^/there is a/ need for 
understanding the multiple frames of reference 
with which actors perceive knowledge and the dis- 
crepancies between the frames of reference of 
knowledge producers and kn^ledge users (Weiss 
and Bucuvalas, 1980:302). * 
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Finally , other explanations place primary emphasis on v 
phenomena generated by social systems. Janet Weiss argues that 
attention to "relationships between the structures of poliqy 
making and the 4venues by which social science may participate in 
policy debates permits analysis of both the overall level of 
social science use that sectoral arrangements seem to support and 
the particular kinds of use that sectoral arrangements encourage" 
(J. Weiss, 1979:439). Caplan provides a similar but more general 
explanation by stressing generated phenomena that jilso operate 
through social systems. In Caplan' s words, "social scientists 
would be well advised to pay particularly close attention to the 

utilization theories that stress the lack of interaction between 

i 

social scientists and policymakers as a major reason for nonuse" 
ICaplan, 1979:461; see also Caplan," Mortison, and Stambaugh, 1975 
and Poppen, 1978). 

A TYPOLOGY OF THEORETICAL PERSPECTIVES 

The intersection of definitional and explanatory dimensions 
4 outlined in the preceding sections yields a basic typology for 
classifying theories of knowledge use (Figure 1) . The product 
of the typology is^ four ideal types that capture basic properties 
of contemporary theories. In Figure 1 these ideal types have 
been labelled with terms that describe the essential features of 
competing definitions (explananda) and explanations (explanantes) , 
as these have been discussed so far, rather than the disciplinary 
orientations (e.g., "philosophy of science" or "sociology of know- 
ledge") or preferred metaphors (e.g., "two-communities theory") 

2<ti 
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of their authors. By treating these four theoretical perspectives 
as ideal types, and by attaching labels that describe their most 
basic theoretical prop^rtie^ may circumvent some of the con- 

ceptual entanglement? .th^t now characterize * the field. 

^\ • • < I 

One major ideal ^typical theory of knowledge use may bfe 

designated "interactional imperativism," since it defines know- 
ledge use in primarily objective (interactional) terms and offers 
explanations that stress phenomena that are imposed on users 
(imperative). The work of Knott and Wildavsky (1980) closely 
approximates "interactional imperativism," as ye have character- 
ized tjiis perspective, since their theory of dissemination and 
knowledge use holds that the overt and observable utilization 
behavior of policymakers is a function of the availability of 
"authentic" knowledge provided through "natural" processes. 
Specifically, 

...premature dissemination in tlie absencte of knowledge 
contributes to information overload, thus making dis- 
semination a causa- of underutilization rather than a 
cure. Dissemination should not v substitute for supply 
when natural processes are cheaper and more effective 
... Three difficulties stand out: pretensions to 
knowledge, which make many dissemination efforts pre- 
mature; artificiality (or the Newcastle syndrome) sub- 
stitutes unnecessarily for the natural spread of know- 
ledge; and the-Mathew effect in Which those policy 
makers who are^the most in need of dissemination are 
also the ones /who are the least able to process and 
apply what is)sent (Knott and Wildavsky, 1980:573 
574) ' - 7 

A second ideal-typical^erspective, "cultural imperativism," 
defines knowledge use in essentially subjective (cultural) terms 
and provides explanations thafc stress phenomena that are imposed 
on users (imperative) . An impressive recent treatise on know- 
ledge and knowledge production (Machlup, 1980) approximates 
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cultural imperativism, as we have defined it, since the emphasis 
is on a "knowledge industry" constituted by given structures of 
production and occupational specialization which create new and 
diverse forms of knowledge. While "for most parts of the pro- 
duction of knowledge no possible measure of output can be con- 
ceived that would be logically separate from a measure of i^iput" 

(Machlup, 1980:225), the class of imposed phenomena used to 

* 

explain knowledge and its use are based on 

an industry approach to the total value of the output _ 
of all firms, agencies, departments, etc., that gen- 
erate or ^disseminate knowledge, and an occupation 
approach to the amount or value of the input of know- $ 
ledge producing labor (Machlup, 1980:227). 

A third ideal-typical perspective, "interactional construct- 
ivism, ,f defines knowledge use in primarily objective (inter- 
actional) terms and supplies explanations that stress phenomena 
that are generated by users (.constructed) • Interactional ^con- 

. . / 

structivism is evident in explanations that stress "the sym- 
biotic relationships among policy actors, knowledge, and the 

political and institutional context of policy making" (J. Weiss,, 

/ 

1979: 456-57):. ,The use- of knowledge . by policymakers 

cannot be understood independently of the processes 
for making, public policy. These processes provide 
the critical organizational and political contexts 
that shape policy makers' nfeeds for outside exper- 
tise; create organizational channels of information 
flow that routinely bring (or fail to £ring) idea£ 
and evidence to policy makers' attention and define 
problems,, decision situations, policy alternatives, 
and solutions in ways that . determine which evidence 
is considered to be relevant (J. Werss, 1979:439).' 
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The fourth and final perspective in our basic typology is 
"cultural constructivism, " an ideal-typical theory best repre- 
sented by thfe work of Knorr (1981). Cultural constructivism 
defines knowledge .use in essentially subjective (cultural) terms 
and offers explanations that stress phenomena that are generated 
by users (constructed) . For Knorr, contexts of practical action 
are radically under-determined by generalizable standards or 
rules. As such, practical action is indexical: 

...rules and decision criteria, and more generally 
definitions of the situation, are interpreted in 
context. . . it the concrete, local translation 
of rules or decision criteria which determine 
the selections that are made, and which sub-t 
sequently shape the outcomes of these select- 
ions (Knorr, 1981). 

Clearly, these four ideal- typical theories 0 are highly gen- 
eral and abstract; they do not capture particular assumptions, 
propositions, or hypotheses- put N forth in alternative theories of 
knowledge use. For this reason it is desirable to differentiate 
the basic typolcjgy into additional subclauses of explanations 
(Figure 2) . This yields an expanded typology based on additional 
distinctions among knowledge-specific, user-specific, and social 
system-specific properties of imposed and generated explanations. 
The- product of this expanded typology is twelve ideal-typical 
perspectives of knowledge use. These twelve perspectives are 
illustrated below w£th statements drawn from the work of leading * 
contributors to theory and research on knowledge use. & 
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The Principal Cognitive Relations that > 
Define Knowledge Use Are: 
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Psycho-Interactional Imperativisra 

Psycho-interactional imperativism defines knowledge use pri- 
marily in objective (interactional) terms and provides explan- 
ations that stress phenomena which are imposed on users (imper- 
ative) via given psychological properties of users thdtnselves. 
Mitroff and Mitroff (1979) exemplify psycho- interactional imper- 
ativism insofar as their effort to define and explain/ knowledge 
use! in terms of transactional analysis (TA) and Jungian person- 
ality theory, respectively, is based on an "external" explan- 
andum and an "internal" explanans. in order that effective know- 
ledge utilization occur 

it is desirable that the parties be as different 

on their Jungian profiles as possible* On the 

other hand, it is critical that they come as 

close as possible to functioning in the adult 

mode /posfted by transactional analysis/. Dif- [ 

ference is called for in Jungian terms so that • 

both parties can challenge their assumptions 

about what form and type of knowledge is needed 

so that they can broaden their perspectives . / 

adult functioning is required so that they can 

benefit from their differences (Mitroff and 

Mitroff, 1979:214). 

Psycho-Cultural Imperativism 
Psycho-cultural imperativism defines knowledge use primarily 
in subjective (cultural) terms and supplies explanations that 
emphasize phenomena which are imposed on users (imperative) via 
given psychological properties of users themselves, Campbell's 
"evolutionary epistemology" comes very close to psycho-cultural 
imperativism insofar "as this "descriptive epistemology" — that 
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is, an epistemology descriptive of man as knower—seeks to 

explain the evolution of science and other cultural artifacts 

in terms of psychobiological processes of adaptation and natural 

selection. In Campbell's words, 

...man has evolved from some simple unicellular or 
virus-like ancestor and its still simpler progenitors. 
In the course of that evolution, there have been J 
tremendous gains in adaptive adequacy, in stored 
templates modeling the useful stabilities of the 
environment, in memory and innate wisdom. Still * 
more dramatic have been the great gains in mech- 
anisms for knowing, in visual perception, learning, 
imitation, language and science. At no stage has 
there been any transfusion of knowledge from the 
outside , nor of mechanisms of knowing, nor of 
fundamental certainties (Campbell, 1974:413) . 



^ Epistemo- Interactional Imperativism 

Epistemo-interactional imperativism defines knowledge use 
primarily in objective (interactional) terms and offers explan- 
ations that emphasize phenomena which are imposed on users 
(imperative) via given epistemological properties of knowledge. 
This perspective has already been illustrated with references to 
the ideas of Bernstein and Freeman (1975) , Wildavsky (1979) , 
and Knott and Wildavsky (1980). A variation of this perspective- 
but one that assumes the ubiquity of "knowledge" rather than 
its scarcity or nonexistence — is the "bureaucratic power" model 
of Rourke: "Bureaucratic power thus reflects the technological 
revolution and the growing influence of specialized knowledge 
in modern civilization" (Rourke, 1972) . 
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Epistemo-Cultural Imperativism 

Epistemo-cultural imperativigm defines knowledge use pri- 
marily in subjective (cultural) terms and supplies explanations 
that stress phenomena which are imposed on users (imperative) 
via given epistemological properties of knowledge. Toulmin's 
evolutionary theory of scientific development, based on the 
analogue of population genetics, explains the growth of kaow- 
ledge in terms of competing intellectual variants (e.g., theories) 
carried by scientists in a process characterized by selective 
diffusion, selection, and retention (Toulmin, 1972) . Popper's 

/evolutionary epistemology , which posits a natural trial-and- 

t ■ 

error process of selecting theories whose epistemological proper- 
ties make them more or less fit to solve empirical puzzles, also 
exemplifies epistemo-cultural imperativism (Popper, 197^3; 
Campbell, 1974:436). ( 

Socio-Interactional Imperativism 

Socio-interaotional imperativism defines knowledge use pri- 
marily in objective (interactional) terms and offers explanations 
stressing imposed phenomena that operate via given character- 
istics of social systems. Among the many illustrations of socio- 
interactional imperativism is the "bureaucratization erf inquiry" 
model put forth by Rich (19^9, 1981). In contrast to perspect- 
ives that stress characteristics of knowledge as a key factor 
in knowledge use, the bureaucratization of inquiry perspective 
assumes that % / 
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the characteristics of knowledge aije a necessary 
but not sufficient condition . in accounting for 
the application and utilization of scientific 
knowledge. This view of the problem stipulates 
that the bureaucratization of the knowledge, 
inquiry system — the production, process, and 
use of information — is the critical variable 
for understanding levels of utilization and 
' nonutilization. . . "Bureaucratization" refers 
to issues of internal agency control and owner- 
ship as superseding all other considerations* . . 
Thus, according to this perspective, one would 
affect change in the knowledge inquiry system 
by influencing bureaucratic rules and procedures-^- 
not knowledge and/or policy/sector specific 
characteristics (Rich, 1979:328-329; Rich, 1981). 



SoCio-Cultural Imperativism 

Socio-cultural imperativism defines knowledge use primarily 
in subjective (cultural) terms and supplies explanations that 
stress phenomena which are imposed on users (imperative) via 
given characteristics of social systems. An appropriate illu- 
. stration of socio-cultural imperativism is Wilensky a s treatise 
on "organizational intelligence" (Wilensky, 1967) . Addressing 
"institutional threats to the reasoned use of knowledge," 
Wilensky explains variations in organizational intelligence by 
emphasizing phenomena imposed on users via given characteristics 
of complex organizations and their environments: "The more an 
organization is in conflict with its social environment or depends 
on it: for the achievement of its central goals, the more re- 
sources it will allocate to the intelligence function and the 
more of those resources will be spent on experts..." (Wilensky, 
1967:10). , 
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Psycho- Interactional Constructivism 

- . y 1 

Psycho-interactional constructivism defines knowledge use 
primarily in objective (interactional) terms and offers explan- ', 
ations that emphasize phenomena which are generated by users (con- 
structed) via dynamic psychological properties of users themselves. 
Etheridge's regent overview (1979) of theory and research on 
government learning outlines, among other perspectives, a dynamic 
"bureaucratic dependency theory": 

Dependent people tend to be fearful of, and to resist, 
change... They may have a greater need for leaders 
to define reality, lead, and think for them... be 
more prone to stress, complain without being con- 
structive, tend to wait for other people to solve 
problems. They may also be prone to symbolic poli- 
tics rather than substantive problem- solving. . . 
dependency can be induced and sustained by bureau- 
cracies and need not be solely a personality trait 
(Etheredge, 1979:38-39). 

- •» . 

Psycho-^Cultural Constructivism m ^ 

Psycho-cultural constructivism defines knowledge use pri- 
marily in subjective (cultural) terms and supplies explanations 
that stress phenomena which ar*e generated by users (constructed) 
via dynamic psychological properties of users themselves. The 
definition of government learning proposed by Etheredge, 1979:4, 
while it includes "behavioral effectiveness" as an objective 
(interactional) property of knowledge use, emphasizes "increased 
intelligence and sophistication of thought..." In offering 
explanations of knowledge use as "government learning" Etheridge 
relies heavily on theories of active learning created within 
developmental psychology: 
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Developmental theorists. see people* as fund- 
amentally seeking for — and often achieving — 
further qualitative personal growth* or develop- 
ment, an achievement which is partly facilitated 
or blocked by their environment. This develop- 
mental, so-called rationalist, tradition- holds 
an active, constructionist view of learning, ' 
believes there are innate capacities and pre- 
dispositions of the mind to make independent 
sense out of the world, capacities for active 
internal processing, self-reflection, and 
/qualitative transformation of understanding 
,and competence autonomous from external hedonistic 
incentive systems (Etheridge, 1979:27-28). 



Arguing that organizational learning can be indexed to Individual 
learning — mainly by focusing on "the intelligence and sophisticat 
ion of thought which informs decisions, policies, and programs" — 
Etheridge nevertheless punctuates several special requirement^ of 
government learning: adequate internal learning systems; consult 
ative and adversarial procedures involving external critics; 
organizational memory; and "the embodiment of new understandings 
or revised policies in the action and understanding of people 
throughout the organization" (Etheridge, 1979:9-10). 



Epistemo-'Interactional Constructivism 



Epistemo-interactional constructivism define^ knowledge use 
primarily in objective (interactional) terms and provides explan- 
ations that stress phenomena which are generated by users (con--' 
structed) via dynamic properties of knowledge. Weiss 1 platform 
for a new social sociology of knowledge applications — that is, 
a sociology of knowledge "turned upside dpwn" (see Holzner, 
1978:8) — is principally concerned with the effects of frames of 
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reference on decision making (Weiss and Bucuvalas, 1980:25). 
Identical or closely related perspectives are evident in work 
on "knowledge systems" (Holzner and Marx, 1979) , "theories-in- 
use" (Argyris and Schon, 1974; Zaltman, 1977), "strategic assump- 
tional analysis" (MasOtf'and Mitroff, 1981), and "reforms as argu- 
ments" (Dunn, 1981) . 

Epistemo-Cultural Constructivism 

Epistemo-cultural constructivism defines knowledge use pri- 
marily in subjective (cultural) terms and provides explanations 
that emphasize phenomena which ar&ajSnG»ated by users (con- 
structed) via* dynamic properties of knowledge. Laudan' s theory^ 
of scientific progress approximates the meaning of epistemo- 
cultural constructivism insofar as its principal focus is on the 
development of new scientific theories as a consequence of com- 
peting rational standards or rules for certifying and challenging 
the adequacy of knowledge claims (Laudan, 1977) . While scien- 
tific theories are undeif determined by data, this does not entail 
the conclusion that knowledge claims are properly explained solely 
or even primarily in terms of externally imposed "sociological" 
factors (Laudan, 1981) • 

Socio- Interactional Constructivism 

Socio-interactional constructivism defines knowl^gkp^use 
primarily in objective (interactional) terms and provides explan- 
ations that stress phenomena whLcfh are generated 4ti users (con- 




structed) via dynamic properties of t\xe social systejps of which 
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users are members. > Lindblom and Cohen (197 9) equate knowledge ' 
use with "social problem solving," that is, "processes that are 
thought to eventuate in outcomes that by some standard are an 

improvement on the previously existing situation For us, 

•solve* doex^not require an Understanding of 'the problem 1 but 
only an outcome. .." (4) . "Problem solving" defined in this way, 
is then explained primarily in terms' of social proc^s^es that are 
interactive, pluralistic, and incremental (sGe tindbloV 1980). 
In their words: * / 



4 



Information and analysis constitute only one route * 
among several to social problem solving. % a great 
deal of the world's problem solving is and ought to 
be accomplished through various forms of social 
interaction that substitute action for thought, 
understanding, or analysis. r Information and 
analysis are not a universal or categorical pre- 
scription for social problem solving. In acidic 
tion, PSI /Professional Social Inquir^7 is only 
one among several analytic methods, because other 
forms of information and analysis — ordinary know- . 
ledge and casual analysis foremost among them — 
are often sufficient or better than PSI for social 
problem solving (Lindblom and Cohen, 1979:10). 

Socio-Cultural Constructivism 'xj 

The last ideal-typical perspective yielded by our expanded 
typology is socio-cultural constructivism. Socio-cultural con- 
structivism defines knowledge use primarily in subjective (cul- 
tural) terms and provides explanations that stress phenomena which 
are generated by* users {constructed) . via dynamic properties of 
social systems. This perspective is evident in much research on 
"conceptual " uses of social science knowledge by policymakers 
(see C. yeisa, 1977:15-16), including so-called "enlightenment" 



O Or 

ERIC • 2? j 



-29- 



models of knowledge use (e.g., Rose, 1977; Cohen and J. Weiss, 

1977) and those which stress "ideology" in the classic sense of 

false consciousness (Mills, 1959; Habermas, 1970; Gregg et al . , 

1979) . Representative expressions of socio-cultural construct- 

ivism, as that perspective is characterized here, may be found 

in Rein and Schon' s overview of problem setting in policy research 

(1977) and in general treatises on the role of social science in 

public policymaking (e.g., Rein, 1976; Rein and White, 1977^. / 

Rejecting $ narrow and instrumentally oriented conception of 

problem solving, Rein and Schon propose instead, that the process 

of policy development 

. % . is essentially about a process of problem setting ; 
it is concerned with developing new purposes and new 
interpretations of the inchoate signs of stress in 
the system that derive from the pas,t.» 

Policymakers use research as .an instrument to legitimate 
action in perpetual striving for "consensus of belief 
and for organization of th^v fine structure of ^govern- 
ment action; policy may influence the research agenda 
more than research influences the direction of policy 
(Rein and Schon, 1977:235, 236). 

In summary, there are at least twelve ideal-typical perspect- 
ives of knowledge use now available to enhance our understanding 
of the role of policy research in social problem solving. Each of 
these twelve perspectives, it should again, be emphasized, is an 
"ideal" or "extreme" type in the sense that it represents a one- 
sided accentuation of a particular set of standards for assessing 
knowledge use which is Ipaxt of. a wider universe of diverse and 
frequently conflicting views about knqwledge and its practical 
uses.. While the basic and extended typologies described above are 
artificial entities, as indeed all intellectual constructs must 
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be, they capture many of the' most important perspectives now 
prevalent in the field. 

The contributions of different theorists have been used to ^* 
illustrate each ideal type; ideal types, therefore, are not 
intended as methodological generalizations about theorists whose 

own research typically evidences a great deal more complexity 

* ■ + 
\ 

than a given ideal type can possibly capture. Nevertheless, this 
process of strategic typification is a reasoned One" that delib- 
erately circumvents unproductive disciplinary labels (e.g., 
"political, " "economic, " "sociological, " "philosophical") , pre- 
maturely specified distinctions (e*g. , * "conceptual" versus 
"instrumental" use) , and doubtful hypothesis and ^empirical gen- 
eralizations (e.g., "knowledge use is a function of the 'quality 1 
or 'authenticity' of knowledge -and its k availability to policy- } 
makers. ") . Instead, the basic and extended typologies* are in- 
tended to uncover and raise to £ level of explicit consciousness 

J 

the most basic theoretical properties of an emerging but under- 
standably complex and tangled field. ' 

CRITICAL CONSTRUCTIVISM: A METATHEORY 

The twelve theoretical perspectives outlined in the last 
section may be investigated in several different ways, depending 
on what standards of assessment we wish to* employ in appraising 
them. One invest igativ^ strategy is to assess theories in terms 
o£ their empirical content, for example, by aggregating or 
otherwise reviewing available research findings that seem to 
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oppose or support their claims, underlying assumptions, and 

hypotheses (se6 Rogers and Shoemaker, 1971; Rothman, 1974; ^> w 

Bernstein and Freeman, 1975; Yin'and Heald, 1975; Dunn, 1980). 
This strategy, while useful for some purposes, is likely to prove 
highly unsatisfactory as a vehicle for improving theories of ( 
knowledge use. We are. likely to find that available studies are 
essentially incommensurable (see Larsen, 1981) , not only or even 
primarily because there are discrepancies in procedures for 
measuring or "calibrating 11 variables, but because available 
studies reflect unresolved basic theoretical conflicts among 
those who have conducted them. Under .these circumstances it 
- would come as no surprise that we are presently unable to general- \ 

ize any contemporary theories of knowledge use or planned change 

r 

(Bermaft, 1981) . 

Another investigative strategy is to assess theories in ,terms 
of their logical properties, for example, \by examining their 
scope, simplicity, consistency, determinacy, and level of abstract- 
ion (Wallace, 1971; Popper, 1961; Merton,Vl967) . While the 
logical properties of theories are indeed important, 'this stra- 
tegy is inappropriate under conditions where we do not know what 
to jexplain — that is, where we have not yet decided how to define 
knowledge use itself \ Indeed, this formal logical strategy, when 
combined with empirical standards for assessing theories, is it- 
self a theory of knowledge use that parallels in most respects 
what has been described as "epistemo-cultural imperativism. " 
\— - An interest in empirically confirmed and logically consistent \ 
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propositions reflects a special technical definition of knowledge 
and our own preferred mode of explaining why we will (or wij.1 not) 
use knowledge conveyed by a given theory. 

A third strategy of investigation might involve some combin- 
at ion of h^ptorical ai*d social-psychological analysis. Here we 
might focus on the sociohistorical or psychological bases of 
theories, or the wider research traditions of which they are 
part — for example, by examining thfe professional incentive sys- 
tems that promote and retard the acceptance of given perspectives 
or the psychological traits of theorists themselves. Apart from 
, the present impracticality of this alternative, 'we should recognize 

« 

that this strategy would also commit us to particular theories of 
knowledge use — that is, we would necessarily be applying standards 
of assessment that fall within the scope of "imperativist" per- 
spectives that emphasize social system-specific explanations. 

The point is that each of the *four basic perspectives and 
tfreir differentiated subtypes may be used to derive standards of 
assessment for" appraising all other perspectives, including the 
reflexive application of a theory to itself. While this observ- 
ation may seem unnecessarily abstract or convoluted, it also 
yields an important practical insight* Each theoretical perspect - 
ive may be viewed as a theory-in-use that actually affects the 
behavior of diverse knowledge users . Thus, we can take each per- 
spective as a "datum" that constitutes a frame of reference and 
component underlying standards for assessing "usable knowledge" 
(see Zaltman, 1981; Holzner and Fisher, 1979; Dunn, 1981). Here 
we will foqus on the different ways that theorists themselves 
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"construe" knowledge use, rather than on external conditions that 

may seem to make theories imperative. Yet if we wish also to 

assess or evaluate theories, that is, reflect critically on Atheir 

underlying assumptions, we require standards of assessment that 

are^not^also part of the class of standaris\ intrinsic to the" 

/ ■ , / \ 

theories themselves: "Whatever involves all of a collection must 

not be one of the collection" (Whitehead and Russell, 1910:101; 
Watzlawick et al . , 1974:6). In short, what is required is a 
metatheory of knowledge use, a "theory of theories" that is not 
itself one of the collection. 

The remainder of this section outlines the main contours 
of "critical constructivism," a metatheory whose eclectic multi- 
disciplinary origins lie in the sociology of knowledge applicat- 
ions (Holzner and Marx, 1979; Weiss and Bucuvalas, 1980), the 

t 

psychology of pe^onal constructs (Kelly, 1955), the politics of 
interactive decision making (Lindblom and Cohen, 1979) , and the 
philosophy of argumentation and practical discourse (Toulmin, 
1959; Toulmin, Rieke, and Janik, 1979; Willard, 1980; Mason and 
Mitroff, 1981). In pursuing this aim I shall economize by pre- 
senting a series of propositions and corrolaries that are speci- 
fied in terms of users, knowledge*, and social systems, that is, 
according to £he three main subdivisions of the extended typology 
of theories presented in the last section. 
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Users 

1 • Knowledge use is a systemic cognitive relation structured 

7 

by the ways that users anticipate events (Kelly r 1955). 8 ' Know- 

i * * 

ledge use is systemic in the sense that cognitive relations be- 
tween two or more actors are interdependent . Interdependency 

, v. 

renders such terms as knowledge "transfer" (Glaser, 1973) and 
"translation" (Lafcarsfeld and Reitz, 1975) inappropriate; the 
term "transaction" captures the essence of cognitive inter- 
dependency (Zaltman, 1979) . Interdependency is nevertheless a 
property o$ cognitive relations, that is, of joint occurrences 
based on the act or faculty of knowing. For this reason ^pro- 
duction" metaphors that yield distinctions among producers, 
^/ disseminators (linkers) , and users are at best misleading. Cog- 
1 nitive relations are seldom based on spatiotemporal proximity, 
since "enlightenment" (Cohen and J. Weiss, 1977) is rarely bound 
by overt behavioral relations and motor interaction. Yet cog- 
nitive relations "are not formless but structured, the structure 
both facilitating and restricting a person's range of activities" 
(Bannister, 1962:105; Kelly, 1955). The anticipation of events 
is purposive or teleological , oriented towards the construction 
of future reality (cp. Holzner, 1968),. 

1.1. Users differ from each other in their con- 
struction of events (individuality corollary ) . * 
Users differ "not only because there may have 
been differences in the events which they 
have sought to anticipate but also because 
there are different approaches to the antici- 
pation of the same events" (Kelly, 1955). 
. ^ The distinction between "objective" and 

"subjective" behavior-- between "externally" 
imposed and "internally" generated realities — 
is unnecessary and redundant. As a "unit 
act", knowledge use is constituted by a 
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dialectical conjunction of subject and 
object; objective behavior relations 
alone are never representative of know- 
ledge use. Theories of "instrumental" 
use, when- they exclude the construction 
of events, are not theories of knowledge 
use; theories of "conceptual" use, when 
i they exclude events that are constructed, 

are not theories of knowledge use . 

1.2. Users evolve for their convenience in 
anticipating events a /construction system 
or frame of refer enc^ embracing ordinal 
relationships among constructs -(Organ - 
ization Corollary ),. Users, differ not 
only in their constructions of events & 

4 but also "in the ways they organize 

their Constructions of events*" (Kelly, 
1955) . Organized construction sys- 
tems or frames of reference may be 
characterized in terms of their internal 
consistency and hierarchical complexity. 
What are called "reality tests" — that 
is, standards for assessing the ade- 
quacy, relevance, and cogency of know- 
ledge claims (Holzner and Fisher, 
1979; Weiss and Bucuvalas, 1980;, Dunn, 
> 1981) — reflect complex ordinal relation- 
ship3 among constructs. 

1.3. Users choose for themselves constructs 
through which they anticipate the 
greater possibility fpr the definition 
and extension of their frames of refer - 
ence or construction systems (Choice 
Corollary ) . "Definition is the name 
given to the system's becoming more 
explicit and clear cut in constructing 
elements already subsumed; extension 

is the name given to the system's 
becoming more comprehensive so that it 
may subsume new elements" (Bannister, 
1962:107). A researcher's decision 
to maximize internal and/or external 
validity (Cook and Campbell, 1979) is 
' an illustration of the difference be- 
tween definition and extension, 
respectively, as is the distinction 
between "micropositive" and "macronega- 
tive" policy research (Williams, 1971) . 



EMC 



-36- 

r 



1.4. A construct is convenient for the antici - p. 
pation of a finite range of events (Range 
Corollary ) . Users may construe "effect- 
ive" policies versus "ineffective" ones , 
but will not find it convenient to con- * 
strue "effective- versus "ineffective" 
research findings, since research findings 
are .typically outside thte range of con- 
venience of the construct "effective-in- 
effective. " 

1. 5 I A user's construction system or frame of 
reference changes as he successively con - 
/ strues replications of events (Experience 
/ Corollary ) . The constructions ohe places 
/ upon events are "working hypotheses" which 
are put to the test of experience. A 
user's "anticipation or hypotheses are 
successively revised in the light of the 
unfolding sequence of events a construct- 
ion system undergoes..." (Kelly, 1955). 
For this reason, policy problems do not 
stay solved; they are redefined and even 
"unsolved" (Ackoff, 1974). 

1.6. The change in a user's construction sys - 
tem or frame of reference is limifced by 
the permeability of the constructs within 
whose range of convenience the changes ITe 
(Modulation Corollary )""! "A construct sys- 
tem is free or determined with respect to 
its permeability" (Bannister, 1962:108). 
The more permeable a user's superordinate 
cons^«icts--for example, those associated 
with underlying assumptions or a worldview — 
the more likely he will be able to change 
subordinate elements of a construct system 
or frame of reference. Permeable and 
impermeable constructs may be illustrated 
by considering the relatively great flex- 
ibility of theories, as distinguished from 
hypotheses, and of policy problems as com- 
pared with policy alternatives or the 
decision rules used to select them. This 
distinction is also evident in contrasts 
between ill-structured and well-structured \ 
problems (e.g., Raiffa, 1968; Mitroff, 
1974) and problem-solving and problem-set- 
ting models of policy research (Rein, 1976; 
Rein and Schon, 1977). 
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1 • 7 • Users may successively employ subsystems of 
constructs that are incompatible (Fragment - 
ation Corollary ) . ; Incompatible subsystems 
of constructs Are evident only when they are 
directly compared; incompatibilities dis- 
appear when a subsystem is linked to a 
superordinate system. For example, policy 
researchers may contend that the provision ' 
of more jobs to the unemployed will lead 
to riots and, other civil disturbances. 
The appearance of self-contradiction* is 
ba^ed on %n inference from a single sub- 
system according to which more jobs 
create more satisfied people. Once jobs 
and ri^ots are linked to the superordinate 
construct of -"relative deprivation" (the 
perceived gap between rising expectations* 
and fixed capabilities) the apparent • 
incompatibility disappears. Single sub- 
systems of construct^, including various* 
"reality tests," are unreliable predictors 
of subsequent action unless also know 
the superordinate construct systems of 

which they are organized components — V 
namely, assumptions matter (Mason and 
Mitroff , 1981) . 

1.8. To the extent that one user's construct 
system or frame of reference j.s similar 
to that employed by another iiis cognitive 
processes are similar to those of tha ; 
other user (Communality Corollary )" ~We 
Should expect similarities in cognitive 
processes between users, j*ith similar 
construct, systems, even when they exper- 
ience different events, ratfier than expect 
similarities in cognitive processes be- 
tween users experiencing the same events. * 
Thus, what we experience as policy pro- 
blems are products of thought acting on 
external events or environments (Ackoff , 
1974), such that the communality or con- 
flict among users' frames of reference 
governs the structure of policy problems . 

1 • 9 • To the extent that one user construes the 

"construction process of another he may play 
a role in a social process involving the 
other user (Sociality Corollary ) , To cort- 
strue the construction processes of another 
is not necessarily to use identical con- 
structs, or employ the same frame of 
reference, but "to possess constructs which 
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assume another's within their range of 
convenience" (Bannister, 1962:110).' Thus, 
for a social, scientist, to play a role in 
the social processes of a policymaker who 
believes that the enforcement of law and 
order will reduce crime does not involve 
the social scientist in simply adopting 
and translatirjg these constructs into 
a researchable problem, but requires the 
social scientist to emply constructs 
that subsume those of the policymaker — 
for example, by showing that poverty 
subsumes law and order and crime. 



Knowledge 

4 

2. Knowledge- is a cultural artifact established through trans - 
actions among construing users . Although knowledge is knowledge 
to anyone who takes it as a basis for some commitment to action 
(Lindblom and Cohen, 1979) , knowledge is structured by the con- 
struction systems or frames of reference of different users. To. 
the extent that users construe the construction processes of 
other users they are involved in a transactional cognitive rela- 
tion where two or more parties reciprocally affect, through 
argument and persuasion, the acceptance and rejection of know- 
ledge claims. Thus, knowledge is not "exchanged," "translated," 
or "transferred," but transacted among users who constructively 
negotiate the truth, relevance, and cogency of knowledge claims 
(Dunn, 1981) . Hence, to "use" knowledge is to anticipate events 
by construing their replications. 

2.1. The adequacy of a knowledge claim is a function 

of its assimilability within a user's superordi - \ 
nate construction system (Adequacy Corollary ) . \ 
The adequacy or "truth" of a knowledge claim 
depends on the extent to which it may be 
* assimilated within a superordinate construction 

9 systertt, that is, a construction system that 

warrants a claim as plausible by virtue of 
subordinate status within tfre construct system 
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(cp. Toulmin, Rieke, and Janik, 1979) . Super- 
ordinate construction systems may *be causal, 
teleological, empirical, pragmatic, ideological, 
ethical, or authoritative {Dunn, 1981) . 
"Reality tests" (Holzner and Marx, 1979) are 
particular types of superordinate construction 
systems. 

2.2. The relevance of a knowledge claim is a func - 
tion of its range of convenience to users in 
anticipating events (Relevance Corollary) . 
Knowledge claims, apart from their adequacy, 
may lie outside a user's -range of convenience 
in anticipating events. Thus, a causal claim 
abpufe the effects of early childhood education 
on subsequent achievement may be assimilable 
within a superordinate construct system (e.g., 
4 theory of achievement motivation), but lie 
outside the range of convenience of users who 
anticipate socioemotional development of child- 
ren. "Relevance" and "utility" tests (Holzner 
and Fisher, 1979; Weiss and Bucuvalas, 1980) 
are- particular representations of a user's 
range of convenience. 

2.3. The cogency of a knowledge claim is a function 
of the requisite force expected of a construct 
system in anticipating particular events 
(Cogency Corollary ) i Construct systems vary 
in the degree to which they anticipate events. 
When claims are construed as adequate by vir- 
tue of their assimilation into- a superordinate 
construct system,, the construct system may 
lack the requisite force expected in antici- 
pating certain events. Particular events 
may evoke "conservative" or "liberal" expect- 
ations about the requisite force of a con- 
struct system in anticipating events. For 
example, physicians and lawyers differ in 
their presumptions about sickness and guilt 
and the need to treat or protect clients. 
"Cogency tests" establish the requisite 
force expected of a construct system in 
anticipating events . 
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Social Systems 

3 . Social systems are artificial entities whose creation , 
maintenance , and transformation is a function of -knowledge trans - 
actions . The social systems in which knowledge use takes place 
are dialectical entities: "Society is a human product. Society 
is an objective' reality. Man is a social product" (Berger and 
Luckmann, 1967:61). For this reason explanations of knowledge 
use that stress imposed or generated social phenomena are one- 
sided and incomplete representations of one or another comple- 
mentary property of social systems. What we know as "markets" 
and "bureaucracies" are artificial entities whose consequences 
are nevertheless real or natural for those who participate in 
their creation, maintenance, and transformation through know- 
ledge transactions (cp. Knott and Wildavsky, 1980)'. Similarly, 
what we know as "social experiments" or an "experimenting society" 
(Campbell, 1971) are symbolically mediated change processes 
involving the transaction of new social arrangements, rather 
than the discovery of experimental outcomes that are independent 
of the preferences of investigators. Reforms, therefore, are 
better visualized as arguments (Dunn, 1981) than experiments 
(Campbell, 1975, 1979) . Arguments are naturally occurring 
corollaries to research contexts; every user conducts his own 
research act (Willard, 1980).. 

3.1. The performance of social systems in achieving 
collective goals is a function of the degree to 
which shared construction systems or frames of 
. reference successfully anticipate events , 
(Rationality CcyrollaryT ^ Rationality is a 
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property of the construction systems and 
behavior of individual and collective users; 
there are as many types o£ rationality as 
there are construction systems in society. 



or non-rationality to social systems j 
(Lindblom, 1980; Wildavsky, 1979) are pro- 
ducts of the construction systems 'of analysts 
rather than those of users about whom 
analysts generalize. It is in this sense 
that rationality is "situated" (Simon, 
1976) or "indexical" (Knorr, 1981). 

The success of social' systems in anticipating 
events is a function of the hierarchical com- 
plexity, permeability, and integration of col - 
lectively shared construction systems or frames 
of reference (Capacity Corollary ). The success- 
ful anticipation of events depends on the number 
of superordinate constructs in a construct sys- 
tem, their permeability, and integration. Where- 
as rationality is a property of self-construed 
effective behavior or performance in anticipating 
events, collective learning capacity (e.g., 
government learning) is a structural property of 
shared construction systems. In this sense 
"initeased effectiveness cannot; by itself, 
index increased learning" (Etheredge, 1979:5) . 

The freedom to make collective choices that 
successfully anticipate events is a function 
of collective learning capacities (Emancipation 
Corollary ) . Collective construction" systems ~ 
or frames of reference are constituted, in 
part, by theories — that is, "higher order 
abstractions by which we gain freedom from the 
particularities.... The theories comprise 
prior assumptions about certain realms of the 
events. To the extent that these events may, 
from these prior assumptions, be construed, 
predicted, and their relative courses charted, 
men may exercise control, and gain freedom for 
themselves in the process" (Kelly, 1955:22). 
Accordingly, a " rational " society (Habermas, 
1970) , or one where social scientists con- 
tribute to "enlightenment", (C. Weiss, 1977; 
Cohen and J. Weiss, 1977), is an emancipatory 
one. In the last analysis knowledge use is 
about freedom and human control; the "depend- 
ent variable" or phenomen-to-be-explained is 
not instrumental rationality, or even collect- 
ive learning capacities, but emancipation. 
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CONCLUSION 



In an afttempt to untangle the field of knowledge use I have 
constructed an extended typology designed to capture the most 
basic properties of present-day theories of knowledge use. Rather^, 
than side with any particular theory, or an ideal-type of which » 
it is an illustration, I have instead attempted to take each 
theory as a "datum." Since most theories are internally frag- 
mented or underspecified it is difficult to explore/their assump- 
tions (propositions, hypotheses) with the aim of uncovering 
complementarities or new opportunities for synthesis that arise 
from theories themselves. 

The analysis therefore shifted to a metatheoretical level 
where three propositions and fifteen attendant corollaries serve 
as bases for making statements about ideal-typical theories as 
a whole. This procedure satisfies the condition that "whatever 
involves all of a collection must not be one of the collection" 
(Whitehead and Russell). The most important conclusion is that 
the "dependent variab Je" or phenomenon- to-be-explained is not 
instrumental rationality, or even collective learning capacities, 
but emancipation. 
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notes 

1 By theoretical perspective I mean a loosely organized set of 
claims and underlying assumptions that provide "not only a "way T 
of viewing" knowledge use, but also a commitment to define and 
explain the phenomenon in specific ways. I thus avoid, for 
reasons that will become clear, more restrictive definitions 
of theory as a systematically organized and logically connected 
set of propositions designed to explaih 1 or predict some oiass 

of phenomena^ The terms "theory, theoretical perspective," 

alid "perspective^ are used inter changably in the first three 
sections of the paper. • 1 » \ 

2 9 v ■ - • ' 

These questions, along with the methodology used to compare and 

contrast' theories (or - theoretical perspectives), are based on 
Walter Wallace's fine attempts to map the domain of sociological 
theory (Wallace, 1969; 1971)y 

3 ' i 
- Specialised .fields devoted to 'the study of cognitive relatibns 

have also grown out of ' several disciplines, including the socio- 
logy of sdience (Merton, 1973), the sociology of knowledge 
applications (Holzner and Marx, 1#79) , qualitative sociology 
(Schwartz and Jacobs, % 1979) , cognitive psychology (Hayes, 1978), 
and the psychology of science .(Fisch/ 1977) J 

4 - ■ 

The objective-subjective behavior distinction is, ,of course, ^ 
somewhat ambiguous,. One source of ambiguity, as Wallace (1969:7) 
cTbserves, is the technical difficulty of. directly observing 
subjective bfehavior (e.g., knowledge, attitudes*, values). An 
equally important source of ambiguity (one Wallace does not 
mention) is the technical difficulty of determining whether / 
objective behavior dpes or 'does not reflect particular cognitive 
properties — for example, knowledge of particular policy recommend 
ations or the firfdings of program evaluations* For this reason 
Merton* (1949) , Suchman (1972), Weiss (1977), and Rein and White 
(1977) have cabled attention to th§ "latent" goals of policy 
research. In short, it is even more difficult to make .infer-, 
ences about "knowledge use" from objective 'behavior than it is 
from subjective behavior. A solution for this problem may be 
found in a metatheory that posits a -dialectical relation between 
behavior noy?, distinguished as "objective* 1 versus "subjective" , 
"conceptual" versus -"instrumental, " "intellectual" versus 
Asocial, " .and R»gnitive" versus "interactive" (see 'Sect/L 
belcfw) . 



on 4 

\ 



Machlup's definition of knowledge use as a subjective cognitive 
relafti^ri leads him to conclude (erroneously I think) that 
"Operational definition of use is 'practically impossible..." 
(191,9:64) . 
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This quotation, and others like it, should be sufficient to 
convince, ttle doubting reader that the terra "objective' cognitive 
relation" is not a self-evident contradiction. While we may 
disagree with such definitions it is important to recognize 
that many observers assume (most ofterj implicitly) that object- 
ive, observable, and overt, behavior if, indicative of some cog- 
nitive relation — namely, one that is based on the subjective 
act- or faculty or knowing. 

7 Wallace (196§: 11-13) juxtaposes the terms "determined" versus 
"free-twilled," while others (e.g., Rein, 1976) reintroduce the 
classic "nomothetic-idiographic" distinction. Strasser (1976), 
in distinguishing "social-technological" and "social-emancipatory" 
explanations, provides a more' powerful and conceptually rich 
schema that addresses the role of critical or ref lectiv^L^pro- 
perties of explanantes of theories. 



g 

This subsection d^aws t heavily on personal construct psychology 
and'the theor y of "constructivist alternativism" (Kelly, 1355J . 
Some of, Kell^Eg 'corollaries have been, excluded from this present- 
ation. For Intensions and interpretations of Kelly's work, 
including research applications in mental health, marketing, 
and Other areas, ^ee Frost and Eraine (1967), Bannister and Mair 
(1968), and Stringer and Bannister (1979). 
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ABSTRACT 



1 

This paper discusses selected constructs 
of the term "use" and the complexity that 
may be concealed by oversimplistic imagery. 
The way in which knowledge use is construed 
may influence attempts at measurement. 
Improvement /in measurement techniques may 
result from increased differentiation in the 
way we think about use, particularly as 
distinct from impact, as well as from more 
imaginative strategies for measurement. 



Introduction 
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The study of knowledge use' is about to come of age after a long period 
of creative nurture. The field is coming of age for at least the following 
reasons: (1) it is receiving the active attention of an increasing number of 

v. 

serious scholars in many disciplines; (2) it is increasingly recognized as 
an important issue by the knowledge application community consisting of policy 
makers and practitioners- in a wide variety of public and private sector 
agencies; anc*(3) a number of well established areas (such as technology 
transfer, R&D management, innovation adoption and diffusion, policy analysis, 
and so forth) are displaying increasing common foci, which in turn overlap 
with those of concern to people in the knowledge use area thus further enriching 
the intellectual base of the field. 

There exi sts however, a serious brake or inhibitor to the further 
development of the field. The effects of this difficulty are equally important 
for all groups which have a stake in knowledge production and application. 
The difficulty is easily described: We are generally lacking an adequate 
understanding of what is meant by the term "use" even though it refers to one 
of the most if not the most crucial incept in the field (cf. Weiss, 1981; 
Larsen, 1981). This concept is often the dependent variable in both explanatory 
and normative or prescriptive models of knowledge use phenomena. The term is - 
often conceptualized and measured in diverse and frequently poorly specified ' 
ways by different people whether it is treated as a dependent, independent or 
intervening variable. Clarification of this concept is one of the most 
pressing needs in the field (Rich, 1979; Larsen, 1980) 

"Doing Before Knowing" ' \ 

If we are to better understand the phenomena of knowledge use for purposes 
of explanation, prediction or control it is essential that a better understanding 
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be achieved about the nature of the concept of use and its measurement. If 
we do not understand this concept well we will not understand any better 
those phenomena knowledge use itself influences. Neither will efforts made 
to understand variables affecting the use of knowledge be fruitful if the 
ultimate dependent variable is not clearly understood and is improperly 
ijeasured. This is the familiar paradox of conceptualization (Kaplan, 1964) 

t 

found in developing fields.' Good concepts are needed for good theory but 
good theory is needed for good concepts. The paradox can be solved. by a 
process of approximation. However, even this may be difficult or premature in 
a field such as knowledge use. In this instance an approach labelled by 
Charles Jones (1974) as "doing before knowing" may be required. This approach 

> 

involves a careful plunge into concept development and modification. It is 
the approach followed in this paper. 

. A basic assumption of this paper is^that there exists multiple dimensions 
and hence multiple conceptualizations of the term "use." These dimensions 
may not necessarily cluster in a ma/iner which suggests that there is a single 
multidimensional concept. The possibilities range from a large number of one 
dimensional concepts of use to a modest number of multidimensional concepts 
of use, to a single multidimensional* concept.. No matter how the term "use" is 
conceptualized there are OMQgtedjy multiple ways of measuring it. This paper 
will explore possible dimensions of the term "use" and comment on certain 
measurement problems. 

Knowledge-In-Use and Frames of Refer ence 

Let me identify at the outset certain percpecti ves I hold which may 
have influenced the selection here of particular dimensions end measures of 
knowledge use. First, I believe a substantial proportion of all research is 
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largely concerned with exploring systematically and in further detail what 
is already known on the basis of commonsense experience among some relevant 
subset of a population. Research often follows practice. This is obviously 
the case when research attempts to explain observed behavior. It is also 
,the case, however, when, research attempts to improve upon practice or behavior. 
For example, probably some teachers already knew the advantage of individualized 
instruction or the limitation of particular teaching methods prior to their 
exposure to formal research relating to these issues. And, of course, many 
teachers did not. Undoubtedly, many teachers and administrators knew that, 
leadership style affected children's behavior prior to their (the teachers') 
awareness of Lewin, Lippit and White's (1939) famous study of autocratic, 
democratic and laissez-faire leadership styles. The rather rapid and visible 
acceptance of this study in education (Getzels, 1978:482) was probably made 
possible by a relatively widespread intuitive knowledge of one or another 
aspects of this study. Efforts to explore theories and knowle\ige-in-use are 
ways of eliciting in a manner which permits careful theoretical analysis what 
is kno^n dp a commonsense basis. One implication of this perspective is that ^ 
core ideas found in research are already in at least .limited use and thereWe 
the major. impact of research may often be to support certain practices, V 
facilitate their dissemination by providing important and timely validation! 
and to provide more detailed and refined insight into practices. This also means 
that direct linkages between research and practice are more difficult to 
establish. It is paradoxical that the organic relationship between research 
and practice poses this dilemma. This leads to another related perspective. 

My second perspective is that most knowledge has its primary" impact on 
our frames of references coding schema. Again, this is more obviously the 
case for so-cal led theoretical, research.' But I believe it is also the case 
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(although perhaps less obviously! for so-called applied research. A particular 
research finding or even a set of findings produced by se'veral researchers 
should not be expected to have a direct impact on practice." The asking for 
evidence of direct impact reflects a construal of knowledge use which 
accurately reflects only a limited number of circumstances. It does not 
reflect the more freo/uent way knowledge is used. Even though examples of 
direct impact are available (cf. Getzels, 1978:480-489), it is naive to expect 
to find widespread evidence (and therefore to" ask for evidence) of the direct 
impact of research on practitioners. As Caplanet al (1975) suggest in 
reference to policy makers, the importance of a research idea for a policy 
maker "lies in its ultimate integration into his entire perspective on a 
problem. In asking him to cite instances of (research use) he is really being 
asked to atomize his conception of social reality, to take knowledge out of 
•its context, a context without which the knowledge would not have been retained 
in the first place" (Caplanet al, 1975:18). An important implication of 
this observation is that for research to have an impact it must be' reasonably 
consistent with users' frames of reference. Ralph W. Tyler (1978:97), who 
prefers the term "cognitive map" to frames of reference, makes this important 
observation: 

"New research is likely to produce new concepts and generalizations 
or give different emphasis to old ones. Practitioners may hear 
about them or read about them, but they are not likely to 

incorporate them into their maps unless they meet one or both 

/ criteria-consistency with the reality they perceive, and/or providinq 
guidance for more effective performance of their tasks.' The 
criteria of internal consistency and parsimonious explanation, which 
are important to researchers, appear to have little significance 
to the practitioner, and this is often the reason why some 
research reports are given little attention by practitioners and 
evoke no changes in their behavior." 

It is difficult to assess the frequency with which encountered research 
is consistent with our frames of reference. There is good reason to think 
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that only infrequently does research h,„. . 

* research have a major impact on our frames of 

reference (Tversky and Kahneman, 1980; Lord et al 1Q70 „ □ 

• Lora - et al. 1979; and Ross and tepper. 

'MO,. Th,s relates to a third perspective which is that the probability of 
-earch a,teri„ 9 frames of referee is very ,„„. There js . multipljcatjve 

relationship among at least the f„„ow,„g probabilities: The probability that 
research „i „ „.,„ informatlQn whjch ^ . ^ ^ 

Probability that such research or knowledge „i„ be disseminated to potential 
users; the probability of Us being accepted; and the probability of its bein g 
cemented. , As we sha „ see jn the ^ sectjon) ^ ^ ^ 
factors to consider.) 

factors Influencino t.h. U s e of Knq wlert r 

Before examini ng ^^X7 know , edge use jn any ^ u ' 
usefu, to identify several factors which may influence the conceptua,i 2 ati„„ 
and measurement of use ip any one situation. The term measurement is 

MowUdge. , am not referrin g to measured issues related to the determina- 
tion of tne validity of various knowledge use constructs. Before the issues 
of tuning construct validity can be addressed we need to have clearer ideas 
of what the possible constructs are whose validity should be assessed. The 
use made of knowledge is influenced by at least the following factors: 

A. The Went of knowledge. . - 

B. The source of knowledge. 

C. The co^unication channel through „„ jcn knowledge is disseminated 
* B. The means whereby the knowledge was produced. 

k C now^\ nCe ^: 5 r e ta c ? rc n l s to an t c h e Pu'entia, user of the item of 

tiona, environment S^SSEVS/Xg T the ° rgan1 "- 
style of a teacher or school a^nfnr orT "messing 
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F. Motivation to use or not use the item of knowledge. 

G. The original purposes for which the item of knowledge was 
generated. 

« « 

H. The need/opportunity for user "value added." 

I. ' The structure of knowledge (independently of its content). 

J. The perceived nature of a problem or decision to which 
knowledge may be applied. 

It is not within the purview of this paper to explore systematically 
how each possible dimension of use may be .influenced" by the above factors. 
However, since these are important factors they will necessarily be touched 
upon at least implicitly in various ways throughout the paper. They are not, 
however, the main concern of' this paper. 



Some Simple Illustrative Uses of Knowledge 

Before exploring some complicated aspects of knowledge use it will be 
helpful to first illustrate some less complicated uses which might be made- 
of a single item of knowledge. A sample knowledge item might be that an 
"interactive research and development on teaching (IR&DT) approach to educational 
research and development i ncreases in effectiveness when participating teachers 
have had similar past experie nces with regard to innovation developments " 
(Tikunoff, Ward, and Griffan, 1979:18). This knowledge may be used or not'used 
in the following ways: 

1. Persons interested in educational R&D who may wish to try 
this approach may seek as participating teachers only 
teachers who have had similar past experiences. 

2. Persons who may be required (mandated) to use this approach 
contrary to/ their own preferences may deliberately seek as 
participating teachers only those who have not had similar 
past experiences. 

3. An active researcher in education exposed to the finding may 
dismiss it because it is perceived to be far too difficult 
to implement. 
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4. Teachers who may be invited to participate in an IR&DTf endeavor 
may evaluate their probable satisfaction' in terms of whether or 
not they have had similar past experiences: those without 
such past experiences might be more inclined to refuse partici- 
pation given this knowledge. [ 

5. Potential adopters of a particular educational innovation may 
use as an evaluation criterion whether or not the originating 
R&D team had teachers with similar past experiences. 

It is not difficult to identify for each of the five uses just mentioned 
(and there are yet others) a set of underlying assumptions and other circum- 
stances that would make each use justifiable from some normative perspective 
and yet another set of assumptions and circumstances that would make each use 
unjustifiable perhaps fromthe same normative perspective. Whether the use 
made is judged a misuse depends largely upin our awareness of and acceptance 
of the associated set of assumptions and >ci|rc urn's tances ( . 

The^first example might be labeled a positive use of knowledge by an 
R&D manager. Of course, a consequence of employing only teachers with 
substantial past similar experiences may result in the same teachers being used 
repeatedly in R&D efforts. These teachers may or may not be representative 
of other teachers or if representative initially, may lose representativeness 
over time as they acquire researcher skills and perspectives. Moreover, this 
approaches not develop appropriate experiences for other teachers. Thus 
this so-called positive use of knowledge may have negative consequences and' 
therefore could also be considered a misuse of knowledge if not applied 
judiciously. What appears to be a positive use may in- reality involve a 
negative use of knowledge. 

The second example suggests a deliberate misuse of the knowledge. If 
one assumes sinister intentions and nothing else, this is a tenable suggestion. 
On the other hand, the R&D manager may have developed an effective alternative 
system to IR&DT in which the lack of teachers with similar past experiences 



2^ ' 



8 



is an advantage for certain types of innovations. Or, closer to the example, 
the R&D manager may have valid reasons to believe an IR&DT approach to be 
inappropriate for a given task, especially where assumptions based on past 
R&D experiences of teachers may be inhibitory. Thus, to lessen the impactof 
an improper R&D strategy (as perceived by the manager) the manager will 
implement it in a way which renders it least damaging. He or she does this by 
choosing as participating teachers those who because of their limited past 
experiences are likely to bring fewer inhibiting assumptions to the task situa- 
tion. The apparent misuse may thus be viewed as an attempt to correct or 
alleviate the negative consequences of an apparent positive use of the knowledge 
by the mandating authority. 

The third example might ordinarily be considered an illustration of non- 
use. If we happen to think the idea is a very good one and easy to implement, 
we might also be tempted to label this as misuse because of our perceptions 
of the negative consequences of deliberately ignoring a research finding which 
could have beneficial impact., A more charitable posture might lead us to 
conclude that the researcher has at least been enlightened since apparently 
it was not the intellectual quality of the idea but i ts N actionabi 1 i ty which 
was rejected. Or was it? Can we even claim enl ightenment? If actionability 
is a very important truth test for R&D managers (cf. Zaltman and Deshpande, 
1979), this particular item, judged to be inactionable, is not likely to 
become a part of the manager's knowledge inventory. The manager will essentially 
forget it. There is a substantial literature on selectivity in perceptual 
bias which focuses on selectivity in retention and motivated forgetting. This 
research cautions that mere exposure to an item of knowledge is not sufficient 
to claim enlightenment resulting from that knowledge. Moreover, selectivity 
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in recall suggests that biases in retention may result in unenl i ghtenment 
as a result of distortion in the original item of knowledge. 

In the fourth examp?*, people who are the focus of the knowledge item 
are making use of it. Teachers with similar past experiences (as they 
perceive that experience) who decide to participate might be considered to 
be making positive u^e of the* knowledge. Similarly, teachers who decline 
participation because they perceive less satisfactory project outcomes or 
less satisfying personal outcomes ifyy be making positive use of the knowledge. 
But what if this self-selectioit process inadvertently yields an excessively 
homogeneous group of participating teachers when the nature' of the task 
happens to warrant heterogeneity among participating teachers? (Teachers 
with similar past experiences in educational R&D may share other character- 
istics in common as well.) Heterogeneity would produce better outcomes and 
hence more satisfaction among participants. Thus had teachers who self- 
screened out participated, the outcome might have been v &bove their required 
threshold of satisfaction for participation. The actual outcomes with 
homogeneous teachers may be below their required (and anticipated) threshold 
of satisfaction. From an aggregate sense one cannot say either group necessarily 
made positive use of the information although as individuals they may have. 
(The issue of anticipation or expectation will be addressed later.) 



The final example concerns potential adopters of\an educational innova- 
tion who use the item^of knowledge as an attribute of an innovation. The 
"scientific" status of the innovation is partly evaluated b\ adoptefs on the 
basis of perceived use made of other teachers like themselves by the developers 
of the knowledge item (Whitten, 1975; Altback, 1975). Depending on the 
adopter's (e.g. teacher's) disposition toward the use of teachers in R&D 
activity, the use made may differ. A teacher may have legitimate (at least 
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to him or her) reason to dispute the item of knowledge and hence reject an 
innovation developed in a process which relied on this item of knowledge. 
Here, of course, what is being called an item of -knowledge is not knowledge 
for the teacher in question. We are likely to misclasslfy this as non-use 
of knowledge when in fact it is non-use of data. The knowledge actually 
being used by the teacher in question is something to the effect that IR&DT 
decreases in effectiveness when participating teachers have had similar past 
experiences with regard to innovation development. 

A sixth example not specified might refer to derived users (similar 
in concept to the notion of derived demand in economics). For example, if 
the knowledge is put into effect students may benefit (or suffer). This refers 
to an aspect of use which we shall develop later. Another beneficiary would 
be a funding agency which sponsored the original research on yt&DT and may 
require subsequent teaching R&D projects it supports to involve teachers 
with past similar experience in the R&D process. 

Still another use of this finding relates to its citation as supportive 
or nonsupportive evidence concerning user involvement in innovation R&D in 
various contexts, or its use for other purposes such as the use being made of 
the knowledge item in this paper. 

Construing Use 

The following sections will identify several different ways of construing 
the use of knowledge. Each of these construals are different even if they 
are partially overlapping or otherwise closely related. Certain measurement 
issues are touched upon as well. When a "doing before knowing" approach is 
necessary the intertwining of concept and measurement is especially acute. 
The danger of unwittingly allowing the concept to be defined by the manner of 



11 



its measurement must be guarded against. Certainly we do have knowledge 
about the concept pf use. However, the phenomena is very complex and in need 
of considerable "doi ng" enl i ghtened by work in decision theory, cognitive 
psychology, and social change, as well as the sociology of knowledge. 

Others have addressed the conceptual and measurement issues of knowledge 
use. Familiar to all of us is the distinction between instrumental usje in 
which knowledge receives an explicit application and- conceptual use in which 
knowledge primarily affects understanding (Caplan, et al , 1975; Rich, 1977). 
Carol Weiss (1980) has also identified a variety of uses of research among 
federal, state and local officials in mental heal th- agencies. These officials 
found research variously useful in (1) keeping up with the field; (2) providing 
a scientific aura to reports; (3) legitimating a particular position"; and 
(4) iri: shaping their understanding of,the social world they must deal with. 
Judith La>|en (1980; 1979) has focused on alternative types of use which 
include: "revisions of the information; use of selected aspects of the 
information; revision of the information to fit the user's situation; a 
decision not to use the information; de facto non-utilization in which the 
potential user does nothing at all." Larsen has also developed an interesting 
set of indices relating to use which she combines to form a Utilization 
Quotient and a variant called the Utilization Sum. 

Impact on Expectations 

The expression "an idea whose time has> ceme" is an intriguing one. 
What is that makes an idea timely or untimely? Undoubtedly there are many 
valid explanations. . For example, a problem may increase in severity to a 
point where a relevant item of knowledge which has been resisted will be 
applied in an effort to solve the problem. Several illustrations from 



12 



community health programs can be given involving changes from the use of folk 
medicine to the use of more modern medical treatment. The inability of folk 
medicines to halt serious physical deterioration prompts the more drastic 
behavior (as perceived by the sufferer) of using the services of a government 
community clinic. Another case would be knowledge about certain procedures, 
to follow in coping with a disaster or emergency. This knowledge isn't 
practiced until an emergency occurs. A suddenly perceived need td respond 
to a particular educational problem may prompt closer scrutiny of long 
existing ideas. This felt need to respond to a problem enhances the likeli- 
hood that at least some of these ideas will be tried. 

Apart from a change in the perceived severity of a< problem, there are 
other reasons, which may account for the relatively sudden use of knowledge. 
One reason is that a kind of convergent validity occurs. That is, a given 
item of knowledge is reinforced by knowledge items from other sources. This 
consistency in information may enhance confidence in the information (Kahneman 
and Tyersky, 1973^) . For example, certain institutions such as White House 
Conferences or special commissions may serve not to generate new ideas but 
to make "more respectable, plausible and thinkable" ideas or knowledge which 
have been widely disseminated or at least have been in existence for some 
time (Popper, 1970). " The experiences of another school or teacher who has tried 
the knowledge may reinforce the potential user's perception of the quality 
of the knowledge. The original idea may be reinforced by non-user evaluations 
such as reports of researchers who have experimented further with the original 
idea. Approval of the idea may be expressed by key administrators. The idea 
may also receive favorable treatment in the popular press. At some point 
sufficient convergent validity exist/and the teacher or school administrator 
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becomes willing to try the knowledge once other necessary enabling conditions 
(e.g. resources) also become adequate. When such a process occurs among a 
significant proportion of a given population we hear the expression, "this 
is an idea whose time has come." 

What may be reflected in this expression is an alteration in expectations 
concerning the consequences of an idea being applied. Users become moK 
confident in the validity and reliability of the idea (however, they define 
validity and reliability). This suggests that knowledge use may be conceptualized 
and measured in a way which follows Bayesian analysis. The reader may already 
be familiar with the work of Marcia Guttentag and her associates in applying 
decision theory to evaluation research (Edwards, Guttentag and Snapper, 1975). 

Following this approach, knowledge use could be defined as the change 
in confidence a person has in a given item of knowledge. First consider a 
teacher relying on a particular test for diagnosing reading skills. This 
person reads an evaluation of this test which supports the ir use of the test. 
We probably will not observe any change in this teacheV>5 behavior. However, 
if the teacher is more confident in his or her use"of this test then we can 
claim the evaluation report has been used. The teacher feels more confident. 
In this instance the report is used to reinforce existing practices. The * 
measure of use is the change in confidence experienced by the teacher. 
Similarly, if tbe evaluation caused the teacher* to question his or her use of 
the test, then its use would be measured by the loss in confidence even though 
there may still be no change in existing behavior. The difference between 
prior confidence in a given practice or idea and posterior confidence is a 
measure of knowledge use. When we are focusing on knowledge relating to 
current practice, it is likely that the change in confidence will be greater 
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when that item of knowledge reinforces rather" then questions the ftleva-nt 
practice. This approach to defining use stresses the acceptance as opposed 
to the implementation of knowledge. The emphasis is on impact on thinking 
as opposed to demonstrable effects on behavior. In fact, the notion of 
confidence relates more to the likelihood of acceptance and thjus js not a 
direct measure of this view of use. \ 

In another instance a teacher may be considering the use of a different 
diagnostic test and encounter an evaluation of that test which 1 s .sufficiently 
supportive to warrant at least trial use of the test. Here we, would be 
inclined to conceptualize and measure use in terms of actual adoption of the 
test. However, we could still apply prior confidence and posterior confidence 
measures as well. In fact, this would be more fruitful because it begins ' 
to provide an indicator of the threshold level an idea must reach (even if a - 

f 

very context specific threshold) for its "time" to h*ve come. The interesting 
possibility is raised that we might be able to predict the likelihood of 
continued use or, conversely, of a discontinuance of the use of the item 
of knowledge (i.e., the evaluation report's basic message) by the magnitude 
of the difference between prior and posterior confidence levels once we know 
what the critical threshold is. Also, independently of the degree to which 
the threshold is exceeded, the magnitude of the difference between prior and 
posterior confidence levels will have an impact. This is especially likely 
when the prior confidence level was well below the required threshold. A 
significant change within a short time interval raises the possibility of 
dissonance arising. This would be followed by subsequent seeking of confirmatory 
information (Zaltman and Deshpande, 1979). 

When a significant-number of people reach their critical confidence 
level at about the same time we have an "idea whose time has come." This is 
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represented by the first -inflection or take off point on an "S" shaped dif- 
• fusion curve. Thus in proportional terms what constitutes a significant 

number of people -need not be particularly large. 

Unit increments in knowledge may be valued 'differentially. -Tffus we 
^*may have the kinds'bf situations displayed in Figures la and 1?. 



Confidence 
LeveJ 




Critical 
Confidence 
Level II 



Critical 
Confidence 
Level I 
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Knowledge Increment r 
, Figure la. Differential Valuation of Knowledge Units 



Confidence, 
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Knowledge Increment 
Figure lb. Equal Valuation of Knowledge Units 
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In Figure la initial knowledge increments have less impact on confidence than 
to those associated with. the take-off or sharp increase in confidence level - 
(Level I) for a population as a whole. After the next critical confidence 
level (Level II). is reached subsequent increments in supporti ve knowledge may 
have less impact. In Figure lb each increment in knowledge has, the same 
impact on confidence level and hence are valued equally at least up to the 
critical confidence interval. The critical confidence level could be Estimated 
by a diffusion curve in cases where -'the knowledge item has a physical mani- 
festation in the form of a test" ins trume^ book, classroom design, etc. If 
the diffusion curve, were a- classical "S" shaped diffusion curve then the 
critical confidence level associated with the. "idea whose time has come" is 
at the' point where the so-called early adopters are beginning to accept the 
practice. This*is shown in Figure 2. 



Time 




Critical Confidence Threshold 
for Early Adopters 



Number of Adopters 
Figure 2. Traditional "S" Shaped Adoption Curve 

Figure 2 may be reconceptualized as showri in Figure 3. Figure 3 displays 
a third dimension representing confidence level. Thus early adopters (EA in 
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Confidence Level 



Number of Adopters 



Figure 3. Confidence Level for Early Adopters of Knowledge 
, (Note: The Knowledge Increment dimension is 
omitted.) 



Figure 3) are high in confidence, early in adopting relative to other potential 
adopters, and early in adopting relative to the time in which the knowledge 
appeared. A fotrrth dimension representing increments in knowledge should 
also be present but is omitted for visual simplicity. Early adopters would 
probably be represented on the lower end of this dimension approximately 
corresponding to .the point indicated by Critical Confidence Level I in Figure la 

An additional perspective may beV further help. The literature on 
innovation' adoption has, for heuristic purposes, distinguished among several 
different decision making stages (cf. Zaltman and Duncan, 1977; Everett M. Roger 
and Floyd Shoemaker, 1971). A simple unidirectional construal of these 
stages is shown below: 



-Perception ^Motivation ^Attitude Formation ^Legitimation- 



►Trial *Evaluati 



on 




Adoption 
Rejection 




Resolution etc. 



9 

ERIC 



3l»; 



18 



This process primarily describes the effects of accumulated knowledge/about a 
problem and an innovation which addresses that problem. For example, a 
person learns of a problem, becomes motivated to acquire information about, 
possible solutions, subsequently develops attitudes toward the innovation * 
which, if favorable, causes the person to W further reinforcing or legiti- 
mating evidence which may precipitate a symbolic or actual trial. Information 
gained from.this trial is evaluated and "subsequently leads to a decision to 
adopt or reject the innovation. Further information is used to reaffirm 
or possibly question the adoption/rejection decision. 

Knowledge is used at each of these stages. The knowledge or information 
used at each stage may often vary. Additionally, the same knowledge may be 
used differently at different stages. As a certain confidence level fs 
reached with regard to, say, attitudes, a person moves into the legitimation 
stage. When a sufficient confidence level is reached at that stage the trial 
stage is entered and so forth. Only certain stages (e.g. trial, adoption) 
under certkip conditions (when the innovation has a physical form) will have ' 
behavioral manifestations. Most stages and indeed many innovations do not 
have observable manifestations. Most knowledge that is "used" as a person goes 
through this process is used with Regard to the less visible stages. It is 
used to: (1) recognize and understand a problem; (2) become motivated to 
acquire still morlfe knowledge; (3) form attitudes; (4) legitimate the possible 
action; etc. Tn all instances increments in knowledge are associated with a 
series of different and perhaps sequential critical thresholds. It is important 
to point- out that a person may not go through the entire process. A critical 
threshold may not be achieved at any one stage to w^rant further consideration 
of the innovation. In a sense, a negative critical threshold is achieved. In 
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fact, most innovations do not succeed in terms of widespread use or succeed 
only after a long time period. However, pertinent knowledge about the 
innovations is being used even though overt adoption of the innovation does 
not occur. In fact knowledge may be used in very different ways regardless 
of the outcome of a decision. It becomes a nearly impossible task to associate 
a unique knowledge item with the acceptance or rejection of an innovation 
since knowledge impacts frames of reference relating to several different 
socialand psychological activities such as motivation, attitude formation, 
legitimation, resolution and so forth. 

> 

\ ' ' ' 

What If...?' W \ . 

y One. way of measuring use of knowledge is to estimate the effect of its 
absence. First let's consider a simple hypothetical illustration. -Assume a 
particular pharmacological advance has resulted in an effective curative 
treatment for an illness for which no treatment previously, existed'. This 
knowledge advance may be physically manifested by a drug. The impact of that 
knowledge could be measured prospectively by observing the number of people: 
(1) having the illness, ^(2) taking the new drug, and (3) experiencing a 
cessation of the illness. One could also retrospectively assess with reasonable 
accuracy the impact of this knowledge. The question, "what if the knowledge 
leading to the new treatment did not exist?" could be readily answered at 
least in terms of its impact on users' of the treatment. (The issue of use 

p 

versus impact is discussed elswhere in this paper,)- If the new drug were 
preventive in nature we would simply, need the additional estimate of the number 
of people who would likely have*experienced the'ilTness in order to answer 
the "what if" question. 
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An alternative- approach to educational R&D may also be necessary. For 
• many specific R&D efforts, as well as for R&D in general it is difficult to 
.link unique knowledge applications with unique products of knowledge! We may 
know that the linkage is there just as we know attitudes and behavior are 
often linked although establishing the connection empirically^ difficult 
due to observable and unobservable intervening events. A way around this 
difficulty is to rely on probabilistic model's. For example, if it can be 
demonstrated within a given subjectively determined confidence interval, that 
an average of twenty percent (20%) (or 5% or 40%) of all R&D efforts in 
several sectors (each of which ha*? something in common with educational R&D) 
' ultimately receive widespread application within, 'say, a fifteen year tfjW 
period then we might be justified in applying this twenty percent to existing 
educational practices. Thus, we could reasonably suggest that twenty percent 
of all present' practices in education or in specific areas such as testfng, 
organization change, etc. would not exist or would be very different had there 
been no R&D during a past period of time. We probably could not say which 
practices would be affected but only that a certain percentage would be - ' 
affected. > 

The "what if..." notion might be pursued more formally in different 
ways. One way which will.be mentioned only briefly concerns the concept of 
minimax opportunity loss (regret) (Savage, 1951). An opportunity loss is fefie 
cost of regret we may experience by a failure to take the best availabl^ 
action. The "best available action" might relate to the level of R&D funding 
or areas of funding. The'opportunity loss associated with the absence of . 
educational R&D is the regret we experience with regard to educational practices 
which would have been improved had the foregone knowledge been available. 
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The opportunity loss could also be the regret we experience by not selecting 
the most effective test instrument or best diagnostic approach among the 
available alternatives. Where different tests may each have different 
maximum "losses" we would select that instrument which minimizes expected 
losses. Also, within a given area, say educational testing, decisions must 
be made concerning funding levels. Eagh funding level may be'associated with 
a particular maximum loss. Importantly, the loss may not be linearly related 
to foregone -funding support. An agency controlling research funds would fund 
at that level which yields the smallest maximum opportunity loss. Thus 
maximum opportunity losses are minimized. 

We could also construe our "events" so that we assess the opportunity 
loss of not having an item of knowledge or not applying that item of ^owledge. 
Thus for the existing array of educational practices which have an associated 
but not demonstrably connected knowledge base, we could do the following. 
Identify the opportunity loss associated with the absence of each practice 
were it in fact -a result of the associated knowledge produced by educational 
R&D. Kfext we would take several random combinations of practices where each 
combination does not exceed 20% of the entire set of practices,, (or whatever 
percentage of the set we might estimate to be a result of educational R&D). 
Tftis would generate an array of possible regrets associated with not having 
knowledge or in having it but not applying it when it might feasibly have been 
applied. 



Displacement 

When an item of knowledge is accepted and implemented it may be displaced 
by another item of knowledge. For example the knowledge that the involvement 
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of certain types of teachers at all stages of an R&D process might yield 
higher quality innovations under particular conditions may displace earlier 
"knowledge" that teacher involvement may yield lower quality innovation. 
The knowledge that a new method of teaching or testing is superior to an • 
older method may lead to the discontinuance of the older method in favor of 
the new^r method. Thus, implicit in fhe notion of knowledge use is the 
notion of knowledge discarding or displacement,. If an item of knowledge is 
intended to replace another item then we might expect increased use of '"X" 
tp accompany a decreased use of "Y". 

While we might expect measures of the use of "X". and disuse of "Y" 
to be correlated, the correlation may not necessarily be very high. A researcher 
may perceive a particular notion to be a rival hypothesis to another notion, 
test it as such and consequently rule put the other competing hypothesis. 
However, the user of the researcher's findings may not necessarily perceive 
the v two hypotheses as being rival notions. The very notion of- rival hypotheses 
may be questioned by practitioners who are apt to be more sensitive to the 
differential conditions in which different ideas may" work best or which may 
make "competing" hypotheses "compatible" hypotheses^ Or, it may take time for 
the use of "Y" to lessen even while "X" is being used as teachers experiment 
in their own fashion with both "X" and "Y". Or, one.may find a blend of "X" 
and -"Y" being used. This in turn suggests a new concept or knowledge item ("W") 
whose use can't be measured by looking at "X" alone or "Y" alone or evenat 
"X" and "Y" together. It is also possible that the "X" could impact "Y" prior 
to its acceptance. That is, if "X" questions "Y" although itself is not used 
in lieu of "Y" there may be changes in" the degree and way in which "Y" is 
used. This may be more evident by looking at changes in consequences of using 
"Y". These may show up earlier than amy consequences of the use of "X" or even 
the evidence of the awareness of "X". 
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An interesting variant of displacement concerns so-called negative 
findings. An item of knowledge ma^\imply be that "Y" is incorrect or 
inappropriate for particular ourpos'es} We shall label this Y. However, an 
alternative or substitute for/'Y" is not available. The finding Y may be 
accepted and provide enlightenment but the obvious instrumental use of Y, which 
would be stopping the use of "Y" does not occur or at least not to the extent 
which might be expected; An'administrator or teacher may in fact stop using 
"Y" and replace it with an alttrnative which they have developed. In this, 
case Y leads to cessation in the use of "Y" which is replaced by X^X-.^.X 
which represent. the different ideas developed by users in the course of their 
everyday experiences. If Y leads to formal development of substitutes or the 
articulation and consequent use of private theories-in-use (e.g. X-j ,X^ , etc.) 
then Y could be said to have had an instrumental impact in terms of cessation 
of "Y" and the development and use of many alternative ideas. 

; - * 

Curative^ teventive and Combination Uses 
*v " ; 

J Reference was made earlier to the curative and preventive uses of knowledge. 
A curative use of knowledge would be one which addresses in a corrective 
or remedial way a particular recognized problem. A preventive use of knowledge 
would be one which results in a problem not occurring which might otherwise 
occur or prevents an existing problem from becoming more serious. A combination 
use of knowledge is one which has both curative and preventive effects. For 
example, the research finding concerning IR&DT could have curative effects 
for an R&D manager who uses that knowledge by reconstituting an R&D team through 
the replacement of teachers without similar past experiences by teachers who 
have had &ailar past experiences. This would presumably alleviate a problem 
in effectiveness in educational R&D. This same R&D manager could use this 
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knowledge in a preventive fashion when designing a new R&D team for a different 
and new innovation development activity. To. the extent that the reconstituted 
R&D team assumes new activities in addition to existing activities or as 
.prior tasks a*>e completed then the application of the knowledge item about 
IR&DT would have preventive effects as well. It would thus be classified as ' 
having combination effects. 1 

Much research in education and mental health service delivery has 

combination effects. As a validated program innovation is adopted by existinq 

l 

community mental health centers certain existing problems may be alleviated. 
As that same innovation j^^ncorpora ted in a newly establish center it may 
have preventive effects. Some forms of knowledge may be used in primarily 
curative or primarily preventive ways. For example, "knowledge" generated 
by Whit£ House Conferences may be primarily' curati ve in impact in that their 
agendas often reflect concerns with current problems and existing programs 
and alternatives for addressing those problems (Vinyard, 1979). Many 
sophisticated management information systep* and aspects of the social indicator 
field are concerned with predicting problems so -that knowledge having preventive 
effects may be generated or loca-ted and then applied. In fact the knowledge 
used in the design of such systems is used for preventive purposes quite apart 
from the knowledge which is applied to prevent the predicted problem from ' 
arising or worsening. 

Research on victimization poses an interesting example. Significant 
components of this research are of highly questionable validity despite its 
usage for curative purposes. The use (or misuse) of this research may hot 
alleviate but actually aggravate the problems of victims. Efforts to lessen 
the use of existing victimisation research-based on the knowledge that such 
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research is of very questionable validity-are preventive in nature. That 
is, the problem is prevented from becoming more serious by preventing the 
application of inappropriate information. The application of knowledge 
which questions the validity of previous information intended to have curative 
effects but which is found to be damaging constitutes a preventive use of • 
knowledge. 

» 

Measures of Use 

The conceptualization of "use" discussed thus far necessarily involved 
measurement issues. Certain measurement issues will be looked at.more 
explicitly now. However, these also imply certain additional conceptualizations 
which have not been touched upon. 

* • 

Monetary Expenditures / , ^ 

One measure of use is monies expended on the acquisition of knowledge 
when there is a clear manisfestation of knowledge in the form of a service 
or product. Examples would include expenditures for expert advice and for 
books reflecting a particular philosophy concerning educational practice. 
This is a good summary measure for measuring adoption over different time periods 
when the cost of the technology is constant or where changes in cost over 
time may be adjusted accurately. A major limitation of this measure is that 
it does not provide insight into the proportion of potential users of the 
knowledge who are being reached. Nor does this measure permit comparison of 
different knowledge items or permit standardization for organizations or 
sectors having different cost structures. Technological and profit impact on 
the user cannot be readily assessed either. 
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Unit Adoptions 

Jhe number of "units" of an item of knowledge (e.g., a new text, 
laboratory equipment, etc.) adopted by individuals or groups is another 
measure of use. Depending on the knowledge in question, this may be measured 
by a number of different organizations or persons using the process, technique' 
or equipment, the number (or proportion) of instances (e.g., plants/divisions/ 
operations, etc.) tn which a given agency* is using the knowledge, the propor- 
tion of a given sector using the knowledge, the number of different contexts 
which have adopted the knowledge (e.g., higher education, medical industry, 
chemical manufacturing industries, etc.). This measure of use has the 
advantage of reflecting the pervasiveness of use within a given firm, industry, 
or other defined context of use. However, alone it does not necessarily 
reflect continued use of "enabling'! effects which will be described next. 

Gateway Capacity 

Some knowledge has ah enabling function or effect. That is, use of the 
knowledge is significant primarily in that it permits either the use frf other 
knowledge which is critical to an activity^ it stimulates other activities 
which collectively have a major impact on uWs. If the dollar cost of the )' 
enabling knowledge is small or the number of users is small the trie 
impact of the knowledge is likely to be. severely underestimated if these 
measures alone were employed. Often the enabling function differs in different 
settings and hence requires close monitoring of the different settings in order 
to observe them. This is all the more important when the improvements they 
permit are not anticipated. This ability of one knowledge innovation to open 
•the way for other improvements is its gateway capacity. The effect the use of 
an item of knowledge has on the immediate problem it is intended to help solve 
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may be much less than the effects its use has in paving the way for the 
adoption of other solutions to more pressing problems. Dollar and unit 
adoptions generally' do not capture this important dimension of "u$g>" 

Adaptation 

Many knowledge innovations are modified or adapted, sometimes sub- 
s' 

stantially, by users. For example, in a survey of forty-nine social workers 
we focused on the use of information sought with regard to ^critical problem 
of current concern to the social worker. Of the thi rty-ei At&espondents who * 
had found relevant information, twenty-eight persons or seventy-four percent 
indicated that they had to "change" the information to make it more relevant. 
There are further indications that substantial changes were made to make it 
relevant. Approximately forty-five percent of the respondents indicated that 
further^changes were necessary during the implementation of the information. 
Thus, in this sample, there was considerable adaptation taking place with 
regard to the original information.. One could argue that the information 
became knowledge only after the alterations were made. This would be a plausible 
argument from the user's standpoint. However, undoubtedly the producer of the 
original information perceived it as knowledge and from that perspective at 
least a knowledge adaptation occurred. 

Regardless of whether the adaptations occur prior to or subsequent to the 
initial implementation of the original knowledge innovation, the original 
knowledge innovation serves as an enabling device. It is used as the basis 
for a different solution to the same problem or a new solution to a different 
problem. If knowledge use is measured' only by reference to items which are 
unaltered by users then there will be an underestimation of the impact of a 
given item of knowledge. Widening the measure of use to encompass knowledge 
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adaptations is especially appropriate when circumstances of particular 
poetntial users vary greatly implying a high likelihood of adaptation. Thus 
it is important to measure the extent to which the idea inherent in a knowledge 
item is used rather than simply the use of the knowledge exactly as it was 
produced. 

Use vs. Impact 

A distinction was made earlier between use and impact; In this distinction 
"use" refers to acceptance and/or implementation while "impact" refers to 
consequences of acceptance and/or implementation with respect to the purposes 
for which it was accepted (this distinguished impact as discussed in this 
paragraph from adaptability and "gateway capacity). It is important to assess 
^ this type^of impact. Impact may be treated as a component of use. Accordingly, ' 
the "value added" by a particular item of knowledge needs to be assessed. 
Value added itself has many possible dimensions. When knowledge is expressed 
in industrial technology the possible dimensions include: (1) the financial 
savings the new technologies permit in a production process; (2) lower market 
prices; (3) differentiation of products which permU more effective market 
segmentation; (4) entry into new markets with existing or new products; 
(5) enhanced employee safety; (6) favorable environmental, impact; and so forth. 
These and other impacts are not accurately or easily tapped (if at all) by 
n 6ther measures of use. 

Penetration 

Another element ofl impact concerns the fidelity of the adoption. The 
simple^acquisition of knowledge for overt application does not automatically 
imply actual use. Other factors, unanticipated at the time a formal commitment 
was made to acquire the knowledge may cause a delay in its application. 
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.Additionally, initial implementation followed by long term continued use or 
even expanded use should be distinguished, from instances where the knowledge 
is discontinued soon after initial implementation. An item of knowledge having 
a relatively. low cost but.which is used for a significant period of time - 
could be said to have a greater impact than a more expensive item of technology 
which was discontinued soon after initial use. Reliance solely on unit 
adoption or monetary indicators would tend to underestimate the degree of use 
in the former case and overestimate it in the latter case. 

Conclusion ^ ... 

— ■ ■ 

This paper has. discussed selected construals of^the term "use" and 
commented briefly on certain measurement issues. It seems quite evident that 
a given item otj information which might be considered as valid by a designated* 
•potential user may be employed in very different ways. These applications may 
vary across different users or may be applied in multiple ways by a given 
user. Frequently', knowledge use is unobservabie. All concepts and issues v 
addressed In this paper require extensive further elaboration and challenge. 
However, there ar* certain related matters not addressed in this paper which 
are equally in need of examination. For example, the content of knowledge 
clearly has an interaction effect with use. Whether this knowledge is positive 
or negative appears to be important (Locatis,et'al, 1980). Construals of 
misuse and abuse have not been addressed. It is difficult to see how construals 
of use can be understood without better understanding misuse (cf. Cook et al , 
1980). The social context in which knowledge might be used impacts the * 
character of use., as well as the extent of use. This list could be extended- 
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very substantially. Better understanding of the diverse conceptualization 
of the term "use" should be a high research priority. Without a better 
understanding the field of knowledge use may reach an intellectual plateau 
disappointingly early. 
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ABSTRACT 



^This paper contends that argument is a more appro- 
priate analogy for reforms than is experiment. In an 
experimenting society, the outcomes cannot be said to 
be Independent of the preferences of stakeholders in 
social reforms. The change in analogy directs attention 
to the knowledge transactions in which knowledge 
claims are assessed in accordance with frames of re- 
ference constituted by different configurations of 
adequacy, cogency, and relevance tests. 
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REFORMS AS ARGUMENTS 

William N. Dunn 
University of Pittsburgh 

% INTRODUCTION 

The social sciences are an outgrowth of efforts to understand and alleviate 
practical problems through social reform. The development of social science dis- 
ciplines is therefore practice-driven and not, as is mistakenly assumed by those 
who squat in the shade of the natural sciences, a product of "basic" research. 
This myth of the basic-to-applied research cycle, together with derivative mis- 
conceptions about the role of "social engineering," was challenged by Lazarafeld 
throughout his career (Holzner, Marx, and Fisher, 1977). In hia last published 
book he urged once more that we acknowledge the ordinary contexts of practical 
action which continue to drive the social sciences: 



The argument goes that applied research is 
radically different from basic scientific 
work and therefore detracts talent and 
resources from true progress in the disci- 
pline. This implies a false comparison 
with the natural sciences. It is true that 
technical engineers could not succeed with- 
out the knowledge provided by abstract; 
research in mathematics and laboratory 

experiments of the "pure" sciences. But it .- N ) 



a^n^f^ 5 ^"J 11 " Verai ° n ° f thi8 p *P er waa Rented in June, 1980 
at an International Conference on the Political Realization of Social Science 
Knowledge and Research: Toward New Scenarios, a meeting held in memoriam of 
ItnlZ'^l af el * at , the In8titute for Advanced Studies, Vienna, Austria. I 
should like to acknowledge helpful criticisms offered at that time by Thomas 
D. Cook, Burkart Holzner, Karin Knorr, Niklas Luhmann, and Herman Strasser. 
The paper has since benefitted from the comments and suggestions of Pittsburgh 
colleagues-Evelyn Fisher, Bahman Fozouni, Burkart Holzner, John Marx, Alex 
Weilenmann, Charles Willard, and Gerald Zaltman~and from the reactions of 
Andrew Gordon, Duncan MacRae, Jr., Ian Mitroff, and Sam Overman. Finally, I 
am grateful to Thomas D. Cook and Donald T. Campbell who, in a spirit of «en- 

^^^ ti8an8hiP, ? uPPliCd n ° tea * Mterlala . and references that challenged 
many initial assumptions and claims about philosophic and practical implica- 
tions of quasi-experimentation. 
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is misleading to draw an analogy between the 
natural and social sciences. Nowhere in the 
social realm are there unconditional laws and 
basic theories already well established. 
Quite to the contrary, it is the study of 
concrete and circumscribed practical problem- 
areas that has contributed a gopd part of the 
present-day general sociological knowledge. 
Adopting a famous dictum by Lewin, one could 
say that nothing is more conducive to innova- 
tion in social theory £han collaboration on 

J a complex practical problem. 
(Lazarsfeld and Reitz, 1975:10). 

While the social sciences are thus an outgrowth of attempts to understand 
and alleviate practical problems they nevertheless represent more than "the growth 
of ordinary knowledge writ large " (Popper, 1963; 216). The social sciences have 
built upon but also transformed ordinary knowledge, frequently in ways that pro- 
duce unhappy results. For every Authoritarian Personality or American Soldier 
there is at least one Project Camelot, while countless apparently innocuous or 
incompetent applied research efforts have legitimized bureaucratic interests in 
the name of "science." Indeed, the bulk of social science research appears to 
make little if any contribution to improvements in social theory or social prac- 
tice. 

We are therefore confronted by a paradox: Those very sciences that owe 
their origins to practice rarely produce knowledge which enlarges our capacity 
to improve that practice. For many this paradox is resolved by elevating social 
scientists at the expense of practitioners, typically by urging that canons of 
scientific reasoning displace the routines of politics (Bernstein and Freeman, 
1975). For others the paradox dissolves under the weight of arguments that the 
social sciences simply yield less usable knowledge than do various forms of 
Interactive problem solving based on common sense, casual empiricism, or thought- 
ful speculation and analysis (Lindblom and Cohen, 1979) . Here we are urged 
to displace the social sciences with ordinary knowledge that "is highly 
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fallible, but we shall call it knowledge even if it is false. As in the case 
of scientific knowledge, whether it is true or false, knowledge is knowledge 
to anyone who takes it as a basis for some commitment to action" (Lindblom 
and Cohen, 1979:12). 

This radical juxtaposition of science and ordinary knowledge, while punc- 
tuating controversies surrounding the definition of usable knowledge, 
obscures important questions: According to what standards do policy- 

makers, social scientists, and other stakeholders in social reform assess the 
"truth" and "utility" of knowledge (Weiss and Bucuvalas, 1980a)? 

Are such standards properly confined to "threats to validity" or "p^usible * 
rival hypotheses" invoked to assess social experiments (Cook 
and Campbell, 1979)? Can social science theory and research themselves be 
used to investigate the social origins and practical uses of knowledge that 
has been certified on the basis of competing standards (Holzner and Marx, 1979)? 
Can we raise these competing standards to an explicit level of consciousness 
where they may shape a genuinely critical public discourse (Habermas, 1975)? 
Finally, are reforms best viewed as reasoned arguments or debates., that is, 
"critical social transactions" aiming at improvements in knowledge and its 
social uses (Toulmin, Rieke, and Janik, 1979)? 

In responding to these questions this paper offers six related claims. 
First, the metaphor of the "experimenting society" (Campbell, 1969 in 1975b; 
1971), since it is still burdened with residues of a positivist philosophy of 
science, places unnecessarily severe constraints on the range of standards 
available to assess and certify claims about social reform. Second, and in 
contrast, reforms are best viewed as arguments, a metaphor whose roots lie 
in the everyday social interaction of policymakers, scientists, and citizens- 
at-large. When we revisualize reforms as arguments it is no longer possible 

% 
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to make facile distinctions between "science" ancPWdinary knowledge;" nor /ire 
we likely to reach the patently false conclusion that Vnowl edge der^vecTYi 
one or the other source is inherently superior. Third, a "transact ioaal model 
of argument adapted from Toulmin (1958) provides a conceptual framework which 
not only accomodates the experimental metaphor— including "threats to validity" 
and their philosophic justification—but also permits a radical enlargement 
of standards for assessing and challenging knowledge claims. The transactional 
model is therefore appropriate as a central organizing construct for a new 
social science of knowledge applications, since it clarifies and specifies the 
notion of "frames of reference" by providing several classes of "tests" for 
assessing the adequacy, relevance, and cogency of knowledge claims. Fourth, 
and -relatedly, these classes of tests may be transformed into threats to 
usable knowledge, that is, plausible rival hypotheses about' the conditions 
under, which knowledge claims will be accepted as a precondition of action. 
Finally, the transactional model supplies the contours of a critical social 
science of knowledge applications, that is, a social science which uncovers 
and raises to a' level of explicit consciousness those unexamined prior assump- 
tions and implicit standards of assessment that shape and also distort the 
production and use of knowledge. By making such standards transparent and pub- 

0 

lie a critical social science of knowledge applications may contribute to an 
expansion of individual and collective learning capacities and, thus, to 
emancipatory social reforms. 

n 

THE EXPERIMENTING SOCIETY 



Among the many perspectives available for exploring the nature of social 
reforms Campbell's "experimenting society 1 ' (Campbell, 1971) has attracted great 
attention among applied social scientists in the United States (see Salasin, 
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analogies C 



1973). Drawing on analogies to physics, biology, psychology and other labora- 
tory sciences, this perspective is founded on an evolutionary epistemology which 
claims that the growth of individual and- societal knowledge is a consequence of 
trial-and-error learning processed involving successive 'attempts to compare 
hypotheses with experimentally induced outcomes (Campbell, 1959; 1974). This 
evolutionary view, partly based on Popper's natural-selection epistemology 
(Popper, 1959, 1963), claims that the aijn of experiments is to achieve objec- 
tive knowledge by challenging conventional scientific wisdom and current opinion. 
While experimentation is "the only available route to cumulative progress" ii\ 
education and other domains of social reform it is not a panacea for social and 

scientific ills; nor is it an inherently superior substitute for well-tested 

\ 

\ 

ordinary knowledge that has evolved over man^ centuries of trial-and-error 

i 

learning by practitioners (Campbell and Stanley, 1963:3-4). Indeed, the growth 
of ordinary and scientific knowledge is a cumilative product of evolutionary 
changes in human cognitive capacities for caudal reasoning (Campbell, 1974; 
Cook and Campbell, 1979). Causation, therefore, is an inherited property of 
human cognitive evolution and not a special ptize reserved for university people. 

The experimenting society, while cqnditioned by ineluctable changes in 
human cognitive capacities, is nevertheless an active and critical society 
(Campbell, 1971). The experimenting society requires a critical posture towards 
all knowledge, since there are neither essential nor necessary and sufficient 
causes in nature. This critical posture is embodied in the principle of falsi- 
fication, where "not yet disproven" points to the impossibility of ruling out 
all relevant alternative hypotheses (Cook and Campbell, 1979). While all data 
are theory-dependent, thus punctuating subjective properties of all human inquiry, 
the experimenting society avoids epistemological relativism by positing an exter- 
nal reality or "nature" against which hypotheses may be tested, notwithstanding 

A. 
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the Impossibility of fully testing causal claims against that nature. Moreover, 
theories of causatkon are understood in the sense of "nuisance factors" actually 
present in contexts of practice, while the most valid causal influences are 
those involving factors that may be actively manipulated by experimenters. 
Grounded in ordinary language and everyday interaction, this practical and active 
theory of causation does not presume full and complete causal explanations, as 
does basic research in scientific disciplines. Partial explanations suffice. 
Thus, for example, manipulable causes (e.g., a light switch) may be activated 
to produce a desired effect (illumination) without understanding theories of 
electronics or particle physics. ^ 

The metaphor of the experimenting society is therefore an extension of an 
evolutionary critical-realist epistemology and^ot, as some critics ^ould have 
it, a naive emulation of a natural science paradigm based on hypotfJetico-deduc- 
tive methodology (Patton, 1975; 1978). A critical epistemological posture is 
evident in the distinction between "trapped" and "experimental" administrators, 
the latter of whom are urged \to advocate reforms "on the basis of the serious- 
ness of the problem rather than the certainty of any one answer and combine, 
this with an emphasis on, the need to go on to other attempts at solution should 
the first one fail" (Campbell, 1969 in 1975b:35). While acknowledging the 
vicarious, distal, and social^ embodied character of knowledge (Campbell, 1959; 
1979), the experimental metaphor is based on an ontologically realist posture 
that places primary reliance on experimentally induced outcomes that are inde- 
pendent of the desires of reform-minded social scientists and administrators. 
The appropriateness of the experimental metaphor depends in part on our 

success in establishing that social systems in which reforms are carried out 

V 

are analogous to physical aystems in which laboratory experiments are conducted. 
While a major aim of " writings on quasi-experimentation has been to show that 
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laboratory experiments are not feasible in field settings— that "pure 11 experi- 
ments have been oversold and misrepresented — it is also true that the experi- 
mental metaphor retains an <^>jectivist ontological platform appropriate to the 
study of physical systems. 'For example, describing a nested hierarchy of 
evolutionary learning processes of which science is the most developed Campbell 
observes: 

What is characteristic of science is that the 
selective system which weeds out among the 
variety of conjectures involves deliberate 
contact with the environment through experi- 
ment and quantified prediction, designed so 
that outcomes quite independent of the pre- 
ferences of the investigator are possible. 
It is preeminently this feature that gives 
science its greater objectivity and its 
claim to a cumulative increase in the accur- 
acy with which it describes the world. 
(Cafiipbell, 1974:434). ; ] 

The characteristic feature of social systems, as distinguished from physi- 
cal ones, is that they are created, maintained, and changed through symbolically 
mediated interaction (Holzner , m 1969) . Whereas physical systems may be presump- 
tively characterized in terms of a stable external reality that edits experimen- 
tal trials independently of the preferences of ^investigators (Campbell, 1974: 
435), social systems may be characterized (again presumptively) in terms of a 
dynamic external reality that edits and interprets experimental trials 

on the basis of outcomes that are independent of the preferences 
of some investigators but quite dependent on the preferences of others. Social 
systems, therefore, cannot be satisfactorily characterized as either objective 
or subjective entities, or even as both. Social systems, as dialectical enti- 
ties, are more than both : "Society is a human product. Society is an objective 
reality. Man is a social product" (Berger and Luckmann, 1967:61). 
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This dialectical claim does not simply affirm that social systems are 
cultural entities/whose symbolic and self-reflective properties ^et them apart 
from physical ones; nor does it deny that social systems are perceived as 
objective entities by their tfiembe^s. Rather it affirms that knowledge .of 
social systems must, of ntefessity, be based on an understanding of the diverse 
meanings attributed to reforms by stakeholders who participate in the creation. 

V 

maintenance, and transformation of humanly objectivated social structures. 

Social reforms are therefore symbolically mediated change processes the «*dm> 

i 

standing of which requires that we uncover the action-motivating reasons which f 



guide efforts to alleviate practical problems ^ 



Therefore, the case for social experimentation as the only, "truly scien- 
title" approach to reform (Campbell, 1975b: 72) stands or falls oV the persua- 
siveness of the claim that experimental data are not symbolically mediated — 
that is, that experimental outcomes constitute the sole source of knowledge 
that is not determined by the purposes of the experimenter. To make this N 
claim Campbell asks us tc^ imagine experiments as tribal* rituals which are 

meticulously designed to, put questions to "Nature 
Itself" in such a Vay that neither questioners nor 
their colleagues nor their superiqrs can affect 
the answer* The supplicants set up the altar, • 
pray reverently for the outcome they want, but 
do not control the outcome (Campbell, 1979b:198). 
; ) - 

This tribal analogy, instructive because of its simplicity, raises several 

•» * 

difficulties. First, it is unlikely that all tribal cohorts will accept fhe •> 
rule of empirical correspondence as an impartial standard for resolving problem 
that originalljt^created a need for experimentation. In fact, it is not the ' - 
reasoned acceptance of this or any other scientific norm that alone lends 
authority to experimental data. Also relevant are the^diverse social sanctions, 
including disgrace and expulsion, that accompany the process of. competitive 
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- experimental replication in scientific communities. "This competitive scrutiny, 
is indeed the main source of objectivity in sciences (?olanyi, 1966, 1967; Pop- 
per, 1963) and epitomize^ an ideal of democratic practice f in both judicial and 
legislative procedures" (Campbell, 1975b^80), ■ 

The social organization of inquiry Implied by this ideal-typical community 
of experimenters, even if it reflected, the practice of physicists (cp. Kuhn, 
1971), fails to capture the behavior of policymakers, practitioners, social 
scientists, and other stakeholders in social reiterm. The key participants in 
social? reforms share neither the standards of appraisal nor the incentive 
structure of this ideal-typical community. For this reason experimental out- 
comes are unavoidably me^ated by diverse standards of appraisal which are 
unevenly distributed Wmg stakeholders, in reform. * 

Thus, claims about the appropriateness of the experimental metaphor are 

> 

persuasive only if the nature of experimental results automatically foreclose 
options for symbolically mediated interpretation. While we might grant that 
experimental results "certainly are not speaking for one's hopes and wishes 11 - 
(Campbell, 1979:198), neither do they speak for themselves. Thus, for example, 
experimenters might share norms of competitive replication and experience 
disappointment with outcomes that run counter to their preferences, but never- 
theless resist any inference that experimental results actually disconf irm a 
favored theory of reform. A principal reason for this resistance is that even 
well-socialized experimenters: cannot be expected to share the same theoretical 
framework which elicited the choice of the particular (falsified) reform as a 
promising experimental intervention. Social theories, unlike physical ones, 
are difficult to falsify with experimental data because the interpretation of 
such data is mediated by the assumptions, frames of reference, and ideologies 
of social, scientists and other stakeholders in reform. 
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^The presence of symbolically mediated experimental outcomes is precisely 
what is at issue in social reforms whose aim is to' alleviate problems that 
have been described as ill-structured (Mitroff, 1974), squishy (Strauch, 1976), 
-or messy (Ackoff, 1974). Ill-structured problems are those where the main 
difficulty lies in defining the nature of the problem, rather than determining 
through selective experimental interventions the most effective reform to 
alleviate it. Here the primary sources of invalidity are not those first-order 
threats to internal, external, and statistical conclusion validity detailed by 
Campbell and colleagues (Campbell and Stanley, 1963; Cook and Campbell, 1979), 
but second-order threats that call into question the appropriateness of problem 
definitions that create the need for experimental interventions and their 
assessment in terms of standard *(f irst-order) threats to validity. 

Second-order threats transcend or go beyond first-order threats (history, 
maturation, regression, instability, etc.) by providing metacriteria against 
which the formulation of a problem — as distinguished from constituent causal 
inferences that represent a solution within the boundaries of that problem— may 
be assessed and challenged. Second-order threats are sometimes defined as Type 
III errors by juxtaposing the formulation of the wrong problem (E^) to setting 
statistical confidence limits too high (Ej) or too low (Ejj) in testing the null 
hypothesis (Kimball, 1957; Raiffa, 1968; Mitroff and Sagasti, 1973), Since 
threats to validity have been explicitly invoked as a challenge to the error of 
misplaced statistical precision (Campbell and Stanley, 1963:7) it is desirable 
to devise new terms that do not Jtiinge on the dichotomy of errorful calibration 
(Ej and E IX ) errorful conceptualization (E IIX ) . Accordingly, first-order errors 
(E 1 ) involve the choice of the less valid of two or more causal inferences, 
while second-order errors (E 2 ) involve the selection of the less appropriate of 
two or more worldviewp, ideologies, frames of reference, or problem definitions 
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when Instead the more appropriate one should have been selected (Mltroff and 
Sagasti, 1973; Dunn, 1981:109-110). 

Confronted by an ill-structured problem the reform-minded administrator 
or .social scientist might use "multiple measures of independent imperfection" 
(Campbell and Fiske, 1959; Webb et al. , 1966) to ensure that measures are 
responsive to the diverse aims of stakeholders. Further, 

the loyal opposition should be allowed to add still 
other indicators, with the political process Mnd ad- 
versary argument challenging both validity and rela- 
tive importance, with social science methodologists 
testifying for both parties, and with the basic 
records kept public and under bipartisan audit. 
(Campbell, 1975b: 80). 

r 

These adversary procedures are relevant only where stakeholders cannot arrive 
at a common definition of reform. Yet to justify a reform "on the basis of 
the seriousness of the problem rather than the certainty of any one answer" 
(Campbell, 1975at35) begs the question: On the basis of what standards are 
we to assess the appropriateness of the problem? First-order threats do not 
assist in answering this question, since they are relevant and applicable 
only within a given problem frame. Required are second-order threats for 
critically challenging the appropriateness of the problem frame itself. To 
confuse these two levels is to violate a basic logical axiom: "whatever 
involves all of a collection must not be one of the collection" (Whitehead 
and Russell, 1910:101, cited in Watzlawick et al ., 1974:6). 

Thus, while threats to validity provide a critical mechanism for reducing 
the probability of first-order causal errors (E 1 ) , thfcy do not deal satisfac- 
torily with second-order conceptual errors (E 2 ) . The experimental metaphor 
acknowledges the priority of pattern identification over knowledge of details, 
but only at the level of first-order causal errors: 



"Qualitative, comnon-sense knowing of wholes and patterns provides the 
enveloping context necessary for the interpretation of particulate quantita- 
tive data." Thus, the experimental metaphor calls for the integration of 
qualitative and quantitative standards for assessing and challenging know- 
ledge claims. Yet this general plea simply exhorts social scientists to 
recognize the dependence of quantitative on qualitative knowing, for example, 
to recognize that several threats to validity (e.g., history) are based on 
common-sense knowing (Ibid. ; 15) . Even here the loose translation of , quali- 
tative knowing into specific threats to validity excludes or ignores many 
varieties of qualitative knowing, including^ ever al forms of ethical -and / - 
practical reasoning that are highly appropriate for understanding purposive 
social behavior (see yon Wright, 1971). In short, qualitative knowing is 
not explicitly, formally, and systematically incorporated into the critical 
methodological repertoire of social experimentation. 

JURISPRUDENCE AS METAPHOR 

Any metaphor of reform should be assessed according to its capacity "to 
produce satisfactory explanations of the type of events which it investigates, 
rather than its success or lack of success in-getting results by the methods 
of natural science" (Levinson, 1966:144; J>unn and Fozouni, 1976). Because 
reforms are symbolically mediated and purposive social processes aiming at 
changes in the structure and functioning' of some social system, they neces- 
sarily involve outcomes which are valuative as well as factual in nature. 
The success of reforms therefore depends on rationally motivated consensus 
that some projected future social state is both possible and desirable. In 
turn, any applied social science which seeks to critically assess and improve 
the process of reform must address competing ethical as well as explanatory 
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hypotheses (see MacRae, 1976) . For this reason reforms are appropriately 
viewed as a process of reasoned argument and debate where competing standards 
for assessing the adequacy of knowledge claims include, but are not limited 
to, rules for making valid causal inferences. Here the appropriate metaphors 
are drawn from jurisprudence (Toulmin, 1958), J.aw (Levine and Rosenberg, 1979), 
forensics' (Brock et al ., 1973; Bronw, 1976), and rhetoric (House, 1980), dis- 
ciplines in which causal inferences play an Important but non-exhaustive role. 

The appropriateness of the jurisprudential metaphor becomes evident once 
we consider standards for appraising knowledge actually applied in the course 
of social reforms. In the field of evaluation research Suchman (1972) alerts 
us to the pervasiveness of experimental outcomes which are mediated by the 
worldviews, ideologies, and frames of reference of stakeholders in reform. 
Collectively described as "pseudoevaluation," these forms of symbolic media- 
tion include the selective use of data to make a reform appear worthwhile 
("eyewash"), the suppression of data which runs counter to the preferences of 
reformers ("whitewash"), the use of data to subyert a reform ("submarine"), , 
the ritualistic collection of data on a reform for purposes unconnected with 
its consequences ("posture"), and the use of data to delay|ref orm itself 
("postponement") . Similarly, Rein and White (1976) call attention to several 
latent goals of government-sponsored policy research, including the contain- 
ment, subversion, and policing of social reforms. Given the complexity of 
social problems scientifically popular recommendations for improving the pro- 
duction and .use of applied social research— including more rigorous research 
designs, better sampling procedures, and administrative centralization— are 
likely to be marginally effective, superfluous, or mystifying (Rein and White, 
1976:244-50). 
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XS* ese observations on the latent goals of applied social research suggest 
that we should begin with the practice of assessing knowledge claims made in 
the course of reforms, hoping to uncover concepts and standards of assessment 
whi?h later' might be used to develop theories of knowledge production and use. 
This, aim can be facilitated by viewing reforms as reasoned arguments, rather 
than experiments which put questions to "nature itself." Arguments are like 
lawsuits, while conclusions are similar to claims put forth in court. Conflicts 
among stakeholders are analogous to cases in law where disputes are settled by 
invoking standards appropriate to different contexts, for example, criminal or 
c^vil disputes. Whereas the aim of jurisprudence is to study the variety of 
concepts and procedures used to resolve legal claims, the aim of the applied 
•social sciences is to investigate concepts and procedures used to argue and 
settle practical claims. The applied social sciences may therefore be described 
as "generalized jurisprudence" or, alternatively, as "jurisprudence writ large" 
(compare Toulmin, 1958;7). 

The jurisprudential metaphor is particularly appropriate for investigating 
reforms, since the data or' evidence introduced in a given case are only one of 
several elements necessary to make a successful claim. Equally Important are 
the standards of appraisal employed to interpret data. Despite the belief that 
the applied social sciences produce conclusive fact and proof 

they are instead engaged in producing inconclusive evi- 
dence and argument. Problem complexity denies the 
possibility of proof and reduces the pursuit of fact to 
the pursuit of those selective facts which, if appropri- 
ately developed, constitute evidence in support of 
relevant argument. 
(Lindblom and Cohen, 1979:81). 

Argumentation is, therefore, a social process where all data or evidence are 
symbolically mediated. Whereas proof, demonstration, or validation hold that 
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truth is directly and immediately attainable, argumentation sges truth entirely 
as a social construction (Phillips, 1973), as a product of natural social com- 
parison processes: 

Put analogically, arguments are naturally occurring corol- 
laries to research contexts... It is rather as if we were 
to stand back and watch while our subjects framed their 
own hypotheses, selected methodological principles most 
appropriate to the hypotheses, utilized techniques appro- 
priate to both, and conducted their own research act... 
arguments give us more information than other kinds of 
% research. 

(Willard, 1980:9-10). 

The jurisprudential metaphor is closely tied to classical and modern philo- 
sophical traditions where reason serves a practical and critical function in 
assessing knowledge claims. The jurisprudential metaphor 

helps to keep in the centre of the picture the critical 
function of the reason. The rules of logic may not be 
tips or generalizations; they none the less apply to men 
and their arguments — not in the way that laws of psychol- 
ogy or maxims bf method ^pply, but rather as standards 
of achievement . . . 

A sound argument, a well-grounded or firmly-backed claim, 
is one which will stand up to criticism, one for which a 
case can be presented coriing up to the standard required 
if it^is to deserve a favourable verdict" 
(Toulmin, 1958:8). 

The jurisprudential metaphor thus emphasizes that argumentation is a process of 
rational advocacy where stakeholders engage in the competitive reconstruction of 
knowledge claims. This competitive reconstruction, in contrast to the competi- 
tive replication of experiments, leads towards a pragmatic and dialectical con- 
ception of truth where social discourse plays a reflective and critical role in 
producing new knowledge. Knowledge is no longer based on deductive certainty 
or empirical correspondence, but on the relative adequacy of knowledge claims 
which are embedded in ongoing social processes. 
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A TRANSACTIONAL MODEL OF ARGUMENT 

j 

Toulmin has operationalized the jurisprudential metaphor in the form of 
a structural model of argument (Toulmin, 1958; Toulmin, Rieke, and Janik, 1979) 
Extensions of this model to issues of public policy have recognized that the 
growing complexity of social problems demands increased reliance on persuasion, 
rather than formal logical certainty, and have called for systematic learning 
from practitipners as a major component of creative theory building in the 
applied social sciences (e.g., Brock et al . , 1973; House, 1977, 1980; Kelly, 
1980) . In developing a reflective methodology for solving ill-structured 
problems Mason and Mitroff {1981) have linked the structural model to a dia- 
lectical conception of knowledge and have attempted to develop appropriate 
technologies that may be employed by policymakers in concrete settings, 

In his critique of positivistic ethical theories Habexmas (1^75^107^ 
employs Toulmin f s distinction between analytic and substantial arguments to 
argue that the growth of knowledge takes place through the rationally motiva- 
ting force of substantial arguments, that is, arguments which abandon criteria 
of conclusiveness, demons t rat iveness , necessity, certainty, justification, or 
validity and which rely, instead, on rational standards of achievement which 
enhance the persuasiveness of claims in particular social contexts (Toulmin, 
1958 : 234) . Substantial* arguments 

IrT-ba^ed on loglcfiK^nf erences, but they are not exhausted 
in deductive systems of^M^atemptffs^ Substantial arguments 
serve to redeem or to crit^ntze validity claims, whether 
the claims to truth implicit in assertions or the claims 
to correctness connected with norms (of action or evalua- 
tion) or implied in recommendations and warnings. They 
have the force to convince the participants in a discourse 
of a validity claim, that is, to provide rational grounds 
for the recognition of validity claims. 
(Habermas, 1975:107. Emphasis original). 
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Toulmin's model of argument, since it accentuates the critical and socially 
transacted properties of knowledge production and use, is most* appropriately 
described as a transactional model of argument- The transactional model is 
important for the applied social sciences because, first, it provides a visual 
representation or structural schema which may be used to systematically map 
arguments offered by applied social scientists, policymakers, and other stake- 
holders in social reform. Second, the transactional model permits and even 
compels a reflective and critical examination of assumptions which constitute - t 
the worldview, ideology, or frame of reference of stakeholders who advance and 
contest knowledge claims. Third, the transactional model may be extended and 
elaborated to yield a typology of standards, rules, or tests for assessing and 
challenging the "truth" and "utility" of knowledge (see Weiss and Bucuyalas, 
1980a; Holzner and*Wrxi 1979). This same typology also yields a classifica- 
tion of threats to usable knowledge, that is, rival hypotheses about the 
adequacy, appropriateness, and cogency of knowledge claims. Finally, the 
transactional model affirms that processes of. knowledge production and use are 
symbolic or communicative actions involving two or more parties who reciprocally 
affect the acceptance and rejection 6f knowledge claims through argument and 
persuasion. Thus, knowledge is not "exchanged," "translated," or "transferred," 
but transacted by negotiating the truth, relevance, and cogency of knowledge 
claims . \ 

The transactional model contains six elements: data (D) , claim (C) , war- 
rant (W), backing (B) , rebuttal (R) , and qualifier (Q) . Together these elements 
provide a visual representation or structural schema that may be used to map 
arguments. The first triad of elements parallel those of the classical syllo- 
gism; minor premise (D) , major premise (W) , and conclusion (C) . The model is 
nevertheless designed as a challenge to the classical syllogism and other analy- 
tic arguments- For this reason Toulmin introduces a second triad of elements: 
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backing (B), rebu|tal (R) , and qualifier (Q) . The backing (B) , which consists 

of additional data, claims, or entire arguments, certifies the assumption 

expressed In the warrant and Is Introduced only when the status of the warrant 

Is In doubt. In analytic arguments the backings of warrants are tautological 

since they Include Information conveyed' In the claim Itself. By contrast, the 

backings of warrants In substantial arguments do not contain Information con- " 

veyed In the claim (Toulmln, 1958:125). Practical arguments of fered In the 

course of a social reform are seldom If ever analytic: 

\ 

If the purpose of an. argument Is to establish conclusions 
about which we are not entirely confident by relating them 
back to other information about which we have greater assur- 
ance, it begins to be a little doubtful whether any genuine, 
practical argument could ever be properly analytical. 
(Toulmin, 1958:127. Emphasis original). 

The two remaining elements of the structural schema /re the rebuttal and 
• qualifier. The rebuttal (R) performs both a retrospective and anticipatory 
role by specifying conditions under which the adequacy or, relevance of a know- 
ledge claim may be challenged. Finally, the qualifier (Q) expresses the degree 
of cogency or force attached to the claim and is typically expressed with such 
terms as "definitely," "very probably," or "at the one percent confidence . 
interval (p-.Ol)." 

The structural schema provides an explicit visual representation of these 
six elements and their role in making practical inferences, much in the same 
way that symbols used to depict different; types of experimental, quasi-experi- 
mental and pre-experlmental designs provide a visual image of. the role 6f 
independent and dependent variables in making causal inferences (Campbell and 
Stanley, 1963). In contrast to experimental design notation Che structural 
schema surfaces and raises to a level of explicit consciousness the assumptions 
and presuppositions which provide rational backing for substantial arguments. 

ERIC ^ ' 
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This critical function of the structural schema may be illustrated by borrowing 
from Campbell (1975b) a well-known example of quasi-experimental reform. 

Following record, high traffic fatalities in 1955 the Governor of Connecticut 
Implemented a crackdown on speeding violators. After one year there were 284 
traffifc deaths, a 12.3 percent reduction from the record high of 324 in 1955. 
On the basis of these data (D) the Governor offered the following claim: "With 
the saving of 40 lives in 1956, a reduction of 12.3% from. the 1955 motor vehicle 
death toll, we can say that the program is definitely worthwhile" (Campbell, 
1975b : 7,5-6) . ? Figure 1 illustrates the Governor's argument, including suppressed 
warrants and backings and rebuttals based on threats to the validity of causal 
inferences (Campbell and Stanley, 1963; Campbell, 1975b). 

(Place Figure 1 about here) 



i 



9 



33o 



Figure 1 



STRUCTURAL SCHEMA APPLIED TO THE 
CONNECTICUT CRACKDOWN ON SPEEDING 



ata 



A 12.3 percent 
reduction In 
traffic fatali- 
ties after the 
crackdown on 
speeding in 
1955. 



THEREFORE 
[Qualifier — 

(Definitely) 



SINCE 
[wj arrant 

Strict enforcement 
of speeding lavs 
caused traffic fat- 
alities to fall. 
Human life is always 
worth preserving. • 

BECAUSE . 
[ftjacking 

The greater the cost 
of an alternative 
the less likely it 
will be pursued. 
Human survival is 
a self-evident 
moral principle. 



-^jcjlaim 

The crackdown on 
speeding was worth- 
while. 

r 

UNLESS 
(iQebuttal 



1. Weather conditions 
were unusually severe 
in 1956 (HISTORY) 

2/. Mass education pro- 
duced safer driving 
habits (MATURATION) 

3. The 1956 decline 
reflepts random fluc- 
tuations in the time 
series (INSTABILITY) 

4. Publicity produoed 
the reduction 
(TESTING) 

5. Record-keeping 
changed in 1955 
(INSTRUMENTATION) 

6. Many speeding offen- 
ders left the state ^ 
in L956 (MORTALITY) 

7. 1956 was unrepresen- 
tative of the time 
series (SELECTION) 

8. Deaths in 1955 were 
extreme and reflect 
regression towards 
the mean of the time 
series (REGRESSION) 



\ 



SOURCES: Adapted from Campbell (1975b) and Dunn (1931) 
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The transactional model permits and even compels a reflective or critica^ 
posture toward^the presuppositions of knowledge claims, whether practical or 
theoretical. For this reason it transcends overdrawn and facile- distinctions 
between "professional aocMl dJfejuiry and "ordinary knowledge" (Lindblom and 
Cohefi, 1979) , viewing both as potentially ideological in the classic s^nse of 
beliefs which originate in unexamined assumptions. The transactional model can 
also assist in transforming the empirico-analytic and hermeneutic sciences into 
critical ones (see Habermas, 1971), since the model forces the inspection of 
causal and ethical assumption^ 1 , as well as their underlying warrants, as part 
of a social process of interpreting qualitative and quantitative data. 

. Thus, the claim that "The crackdown on speeding was definitely worthwhile" 
might withstand all threats to validity but lack persuasive force. Thus, one or 
more stakeholders may question the adequacy of uoderlying causal assumptions 
("strict enforcement of speeding laws caused traffic Natalities to fall") or 
moral principles ("Human life is always worth preserving") . If further support 
is required certain axioms of economic thrfory ("The greater the cost of an alter- 
native the less likely it will be pursued") might be introduced as backing for 

v I 
the warrant, as might ostensibly self-evident moral prihciples (human survival). 

The claim here is not that these particular axioms and principles dre necessarily 

adequate ones, since they are certain td be challenged by stakeholders who hold 

competing theories, worldviews, ideologies, and frames of reference. The point, 

rather, is t^iat adaptations of the empirico-analytic sciences (e.g. , Cook and 

Campbell, 1979) and extensions of hermeneutics (e.g., Pattern, 1975; 1978) do 

not address such questions in a systematic, critical, and self -reflective manner. 

The transactional model thus accomodates all potentially relevant typ^^of * 

claims and forms of argument. Attention is not confined to descriptive and 

explanatory claims, the standard and exclusive focus of the empirico-anal/tic ' 

° Of 
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and hermeneutic sciences, but extends- to claims which have ethical content inso- 
far as their aim is to evaluate or advocate action/ Further, arguments are not 
limited to a particular causal form (for example, deductive-nomo logical explana- 
tion) but include other forms of causal reasoning such as those represented by 
quasi-naturalistic (historical), quasi-teleological (cybernetic), and teleolo- 
gical (practical) explanations (see von Wright, 1971). This capacity to 
distinguish diverse forms of argument and types of claims clarifies a number of 
methodological inadequacies found in the applied social sciences, including 
the pervasive misconception that "evaluation research" evaluates and the regret- 
table tendency among public policy analysts' to view claims which advocate or 
recommend coupes of action ae emotive or ideological appeals that are devoid ' 
of 'rational content? As Tribe (1972), Rein (1976),^ and MacRae (1976) recognize, 
this tendency reflects the implicit positivistic assumptions of policy analysis 
as an applied social science discipline that ignores or denigrates ethical 
discourse. N 

A related advantage of the model is its capacity to surface and make ex- 
plicit the processes of reasoning used to make knowledge claims. Whereas applied 
social scientists and practitioners frequently suppress steps in the process of 
reasoning from data to claim — for example, i>y claiming/ that coefficients of 
association or so-called "predictive" equations speak for themselves as a 
demonstration of causal patterns or sequences — the transactional model surfaces 
implicit causal and ethical hypotheses alike. This critical function is by no 
means limited to hypothesis-testing, since the model may be used to surface 
paradigms, worldviews, and frames of reference which unite epistepic communities 
(Holzner and Marx, 1979), establish the boundaries of disciplinary matrices 
(Webber, 1980), and distort the definition ot social problems (Gregg etal. , 1979) 
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By distinguishing analytic and substantial arguments the transactional 

'model provides the applied social sciences with a framework and methodology 

*°\ transcendin g pseudo-ethical disputes whose resolution appears superficially 

to lie in greater logical consistency or laetter empirical data. In "analytic 

arguments the major premise or warrant is taken for granted, and the main 

task is to demonstrate that conclusions 6t claims follow from the data with 

deductive certainty. Yet it is substantial arguments, and not analytic ones, 
V 

which characterize the bulk of knowledge claims put forth in the course of 
social reforms. For this reason disputes frequently turn on the presuppositions 
used to back warrants, and not on surface assumptions or data such as those 
found in authoritative moral principles or empirical observations which have 
been validated through inter subjective agreement. 

It is, therefore, insufficient to treat ethical hypotheses solely in terms 
of standards appropriate for analytic arguments, for example, metacriteria of 

! - * * 

logical consistency, clarity, and generality .(MacRae, 1976:90-98) or basic 
postulates of moral reasoning (Gewirth, 1979). Much less is it appropriate 
to, confine ethical discourse to standards of appraisal appropriate for the 
empirical study of. social determinants of knowledge or its applications (Holzner 
and Marx, 1979). The structural model, since' it raises substantial arguments 
to a' level of explicit consciousness, penetrates "the rational content of ethi- 
cal and non-ethical assumptions, which individually and jointly motivate the 
acceptance and rejection of knowledge claims. * 

Finally, the transactional model provides concepts and procedures of 
argument-assessment which are reconstruetable and public. The structural' schema 
may be used retrospectively, to" describe and evaluate different types of claims 
and' arguments, but also prospectively. While it is not possible "to list in 
advance the ingredients of a convincing argument" (Phillips, 1973:178), the 
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structural model can nevertheless be used prospectively to design arguments that 
withstand the diverse challenges or rebuttals which are commonplace in debates 
about reform. Mason and Mitrof f (1981) , for example, report the use of the 
transactional model to conduct an inte ractive dialectical debate among stake- 
holders engaged in resolving probl^Ko^ public and business policy. Their 
interactive computer-assisted methodology not only permits stakeholders to 
attach ordinal plausibility values to each element of an argument and the arg- 
ument as a whole but, >more importantly, enables them to challenge and revise 
assumptions in the course of a reflective debate. A similar dialectical 
methodology, also based on the transactional modSl, ha;^ been used to select 
maximally usable performance measures in the domain of evaluation research 
(Dunn, Mitrof f, and Deutsch, 1981). The aim of these procedures is not to 
"scientize" the model of argument, but to enhance prospects for its efficient 
use, recognizing that the capacity for reasoned debate and reflective under- 
standing is a scarce resource. 

TESTING KNOWLEDGE CLAIMS 

* * 

Claims about social reform are products of frames of reference, that is, 
sets of systemically related assumptions that provide standards for appraising 
knowledge claims. A central component of frames of reference is what Holzner 
and Marx (1979:103-11) call reality or truth tests and whieh Weiss and Bucuvalasi 

4 

(1980a) have investigated in the form of .truth tests andjuWlity tests. Truth 




tests are 



decision points concerning evidence; the grounds for - 
accepting or rejecting truth claims include. . .empiri- 
cal as well as formal rational tests. Pragmatic tests 
rest on proof of workability ... there are other tests 
of varying stringency and reliance on trust or auth- 
ority. 

(Holzner and Fisher, 1979:233). 
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By contrast, utility or relevance tests are decision points concerning the 
delineation of an appropriate domain of inquiry or action. The basis fotr 
accepting or rejecting a relevance claim is the "potential significance of 
an item or iine of inquiry. . .with regard to the inquirer f s cognitive inter- 
ests" (Holzner and Fisher, 1979:233)., 

Concepts of truth and relevance tests pose practically and theoretically 

important distinctions. Nevertheless, it is now unclear how such test* are 

i 

actually distributed among stakeholders in social reforms and, indeed, whe- 
' ther the existence and functions of suck tests may be investigated empiri- 
cally. Available typologies of such tests contain ambiguities that now 
ifcpede directed empirical research. Pointing to the elusive nature of 
"relevance" and "rf&ity," Holzner and Fisher (1979:235) observe that "ques- 
tions remain about the exact manner in which some information comes to the 
attention of a person and how it is sifted." Weiss and Bucuvalas (1980a), 
while calling for a new sociology of knowledge applications, call attention 
to the complexity of issues surrounding the concept of frame of reference 
and remind us of the many conceptual and methodological limitations facing 
those who wish to investigate the impact of truth tests and utility tests 

r * 

on individual and collective decisions. 

The transactional model may be extended to generate a typology of 
knowledge claims and arguments which clarifies and specifies concepts of 
truth and relevance. Knowledge claims may be classified according to the 
explicit or implicit purposes of knowledge claimants or their challengers. 
Brockriede and Ehninger (1960) , drawing from fields of forensics and semi- 
otic, classify claims into four types: definitive, designative, evaluative, 
and advocative. The purpose of definitive claims is to provide knowledge 
about the appropriate definition of^some object (What is it?), whereas that 
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of designatlve claims is to supply knowledge about observed regularities 
(Does it exist?). In turn, the aim of evaluative claims is to provide 
knowledge about the valuej 3f,an event or object (Of what worth is it?), 
* while the purpose of advocatlvi claims is to supply knowledge about policy 
(What should be done?). 

Relevance tests are closely related to these _f our types of claims. If 
relevance tests are decision points concerning the delineation of an appro- 
priate domain of inquiry or action, policymakers and practitioners appear 
to be predisposed towards tests of relevance that reflect an interest in 
knowledge about what courses of action to pursue to resolve problems 
(advocative claims). Discipline-based social scientists, by contrast, are 
generally oriented towards tests of relevance that reflect an interest in 
definitions (e.g., definitive claims about poverty, health, or achievement) 
and in observed regularities in society and nature (e.g., designatlve claims 
about the sources of social inequality). These divergent purposes underlie 
contrasts between policy research and discipline-based research (Coleman, 
1972), distinctions between "macronegative" and "micropositive" research 
bindings (Williams, 1971), divisions within the "two-communities theory" of 
knowledge utilization (Caplan., 1979) , and the pattern of factor loadings 
reported by Weiss and Bucuvalas (1980a) in their study of the frames of 
reference of mental health policymakers. In turn, evaluative claims often 
reflect tests of relevance shared primarily by philosophers and social 
critics, for example, those who see in policy analysis an ideology in dis- 
guise (Tribe, 1972). In each case the purposes of knowledge, as reflected 
in these four types of claims, affirm or diminish the relevance of that 
knowledge to different stakeholders. 

i 

The application of a relevance test does not guarantee that a knowledge 



claim will be regarded as sufficiently cogent 1 or forceful. Tests of cogency 
are dependent on the relative force expected of a claim in particular circum- 
stances. Hence, an advocative claim accepted as relevant to the aims of a 
particular stakeholder will not necessarily be viewed as cogent. Tests of 
cogency, which are an extension of Toulmin's qualifier, are evident in the 
practice of different professions. For example, members of the legal and 
medical professions use similar tests of relevance but different cogency 
tests (compare Holzner and Fisher, 1979:235-36). Members of the legal 
profession typically employ a conservative cogency test (qualifier) fdr 
knowledge claims offered in criminal cases; Defendents are presumed inno- 
cent until proven guilty. By\ contrast, members of the medical profession 
often use a liberal test of cogency for claims surrounding the treatment of 
illness: Patients are presumed to be ill, and treated, until proven other- 
wise. In the first case the problem Is to avoid "false positives, 11 while 
in the second it is to avoid "false negatives/ 1 In other cases cogency 
and relevance interact, for example, when school officials set stringent 
confidence intervals in validating the results of achievement batteries 
which will be used to recommend students for jobs or further education, 
but apply liberal tests of cogency when assessing the effects of existing 
curricula on student achievement scores. 

The appraisal of knowledge claims is not exhausted by tests of rele- 
vance and cogency. Knowledge claims, apart from their relevance, derive 

force or cogency from truth tests. Truth tests may be represented in terms 

v 

of different sets of assumptions and underlying presuppositions used to 

v > 
transform data into claims in a practical or theoretical argument. Truth 

t 

tests are standards for appraising the adequacy of knowledge claims, for 
example, by challenging the causal assumptions which underlie a claim. 
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By contrast, tests of cogency and relevance are standards for appraising the 
requisite force and contextual appropriateness of a claim, respectively. 
Typically, tests of adequacy and relevance are discrete assumptions, standards, 
or rules, while tests of cogency are best represented in terms ,of varying, levels 
of force required of a claim. Knowledge claims which are adequate and cogent 
may b£ irrelevant, while claims that are relevant may lack adequacy and 
cogency alike. For example, government-sponsored program evaluations may be 
regarde4 as relevant to the aims of reform-minded social scientists but 
nevertheless lacking in adequacy and cogency when assessed according to stan- 
dards of research quality generally accepted by social scientists, (Bernstein 
and Freeman, 1975). 

Whereas tests of relevance and cogency appear to be comparatively simple, 
truth tests aire complex. Many options for classifying truth tests are availa- 
ble in the writings of philosophers, anthropologists, sociologists, and other 
sofcial observers? and critics. For example, Peirce's contrasts among alterna- 
tive methods of ^fixing belief" (see Buchler, 1953) and Malinowski's essays 

) 

on science, magic, and religion (Mali^wski, 1948) point to a range of truth 

i \ 

tests used to assess the adequacy of knowledge claims. Similarly, Montague 
(1925) and Wallace (1971) distinguish alternative modes, for generating and 
testing the truth of statements about tfre world, modes which differ along 
three dimensions: the status of knowledge producers; the use of/ approved 
methods to produce knowledge; and the reliance on observation as a check 6n 
knowledge claims. 

The experimental metaphor, as we hflve seen, places primary reliance on 
• the correspondence of claims to a stable external reality and secondary reli- 
ance on procedures for determining the coherence of such claims among multiple 
experimenters, (Cook and Campbell, 1979). Authority is also important, since 
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claims are partly certified on the basis of their having been derived from a 
learning process (scientific experimentation) that is belikved to be unique 
in its penetration of a stable and objective external reality (Campbell, 1974) 
The danger is that (necessarily) presumptive ontological claims about what is 
real or natural may improperly authenticate or certify epistemological claims 
about what is true (Michalos, 1981). The ontology of objectivism, when used 
as a justification or warrant for science, may also result in the denigration 
of ethics on grounds that only science produces corrigible knowledge claims 
(Gewirth, 1960). 

The limitation of these schema is that they do not incorporate a variety 
of potentially important truth tests which reflect alternative modes of 
explanation (von Wright, 1971), different knowledge-constitutive interests 
(Habermas, 1971), and competing standards for assessing ethical knowledge 
(MacRae, 1976). Any provisional classification of truth tests should there- 
fore permit distinctions between naturalistic, quasi-naturalistic, and 
practical modes of explanation, enable distinctions among standards of 
knowledge adequacy appropriate for the empirico-analytic , hermeneutic, and 
critical social sciences, and foster an open consideration of possibilities 
for testing ethical hypotheses. 

(Place Figure 2 about here) 

The following classification of truth tests (Figure 2) attempt^ to build 
upon these diverse concerns with standards of knowledge adequacy. 

In contrast to Brockriede and Ehninger (1960), who employ Toulmin's 
model to classify artistic or rhetorical proofs, we use argument as a unifying 
construct to classify standards of knowledge adequacy. This extension of the 
transactional model proceeds from a recognition that the decisive element of 
most contested knowledge claims is not evidence or data (compare Campbell, 
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CLASSIFICATION OF TRUTH TESTS 



Type of Warrant 
or Backing 



EMPIRICO- ANALYTIC 

— Causal 

— Quasi-Causal 



— Typological 



Representational 



— Analogical 



INTERPRETIVE 
— Teleological 



Qu^si-Teleological 



Function of Truth Test 



A nomic connection (e.g., Boyle's Law) 
certifies that events described in data 
(D) are causally related. 

An apparent nomic-like connection (e.g., 
Toynbee's historical cycles) certifies 
that events described in data (D) are 
causally related. 

A typology (e.g., Jung's Theory of psy- 
chological types) certifies that events, 
actions, or persons described in data 
(D) are members of some class. 

A representational rule (e.g., the 
Central Limit Theorem) certifies that 
events, objects, or persons described 
in data (D) are typical or representa- 
tive of some wider population. 

An analogy or metaphor (e.g., the 
servomechanisms of systems theory) 
certifies that relations among events, 
objects , or persons described in data 
(D) are similar to those contained in 
the metaphor or analogy. 



A statement about individual purposes, 
intentions, motivations, or reasons 
(©•8*> goals of policymakers or social 
scientists) certifies that actions de- 

(if 

scribed in data (D) are causally related 
to such purposes. 

A nomic connection (e.g., Ashby's Law of 
Requisite Variety) certifies that collec- 
tive actions described in data (D) are 
causally related to collective purposes. 
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Figure 

PRAGMATIC 

— Clinical 

— Comparative 



AUTHORITATIVE 

— Personal 

/ 

f 

— Ideological 

— Ethical 

— Methodical 

CRITICAL 

— Ontological 

— Emancipatory 
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2 (CONT'D) 



A symptomatology (e.g., the F-Scale or the 
MMPI) certifies that symptoms described in 
data (D) are indicative of an abnormal or 
normal, deviant or healthy state. 

A parallel case or experience (e.g., soc- 
ialized medicine in the United Kingdom or 
PPBS in the Defense Department) certifies 
that events or actions described in data 
(D) are similar in their effects to those 
of the parallel. ~ 



The achieved or ascribed status of know- 
ledge producers (e.g., gurus, scientists, 
or expert commissions) certifies that 
information described in data (D) is 
accurate, precise, or reliable. 

An established belief or doctrifte (e.g., 
scientism, capitalism, socialism) certi- 
fies that .ideas described \n data (D) are 
orthodox. 1 
( 

A norm, value, or principle (e.g., Rawls* 
principle of justice or the Pareto Cri- 
terion) certifies that actions described 
in data (D) are justified. 

The use of an approved method (e.g., path 
analysis or phenomenology) certifies that 
information described in data (D) is 
accurate, precise, reliable, and valid. 



A presumption about the nature or reality 
of valid knowledge (e.g., objectivism or 
subjectivism) certifies that ideas described 
in data (D) are true or right. 

A presumption about the liberation of human 
potential (e.g., self -actualization, theory- 
Y, homo labor ans ) certifies that ideas de- 
scribed in data|(D) are true or right. 



^1974; Popper, 1961), but the underlying standards of appraisal which warrant the 
transformation of data into claims. Data themselves are rarely conclusive; most 
social theories are therefore radically underde,termined by data (see Hesse, 1980) 

Equally important for our purposes, contexts of practical action appear to 
be radically underdetermin^d by generalizable standards or rules. As Knorr 
(1981) argues, practical action is indexical and indeterminate insofar as 
"rules and decision criteria, and more generally definitions of the situation, 
are interpreted in context... it is the cbncrete, local translation of rules or 
decision criteria which determine the selections that are macte, and which sub- 
sequently shape the outcomes of these selections. " At the same time the under- 
determination thesis, whether applied to theory or to practice, does not entail 
the conclusion that knowledge claims are properly explained solely in terms of 
externally imposed "sociological" factors (Laudan, 1981), since diverse stan- 

4 

dards or ruj.es for certifying and challenging the adequacy of knowledge claims 
may hold as much or more explanatory import as do variables such as social 
structure (see Laudan, 1977). 

Truth tests may be classified according to the general and specific func- 
tions they perform in knowledge transactions. These general functions are 
(1) empir ico-analy tic : knowledge adequacy is certified by assumptions about 
the logical consistency of axioms, laws, propositions, hypotheses, or princi- 
ples and/or their correspondence to empirically observed regularities; (2) 
interpretive : knowledge adequacy is certified by assumptions about the action- 
motivating significance of purposes, intentions, reasons, or motivations; (3) 
[ pragmatic : knowltaige adequacy is certified by assumptions about the effective- 
ness of past experiences in producing desired outcomes in parallel contexts; 
( 4 ) authoritative : knowledge adequacy is certified by assumptions about the 
achieved or ascribed status of knowledge producers, the orthodoxy of knowledge, 
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or the use of approved methods; and (5) critical : knowledge adequacy is certified 
by assumptions about the consequences of such knowledge* in emancipating indivi- 
duals and collectivities from unexamined or tacit beliefs that Impede the 
realization of human potential* 

Tests of truth, relevance, and cogency are distinct but interrelated stan- 
dards for appraising knowledge claims. These three general classes of tests, 
together with specific variants, govern the adequacy, appropriateness, and 
requisite force of knowledge which offers solutions t;or practical problems. 
Truth, relevance, and cogency tests ai>e potentially independent, althoughjthe 
force of a knowledge claim (cogency test) frequently depends^on prior assess- 
ments of relevance and adequacy. The reverse is generally not true, since 
various tests of cogency (for example, tests of statistical significance) 
seldom establish the relevance or^rfequacy of* knowledge claims. These genera- 
lizations and the typology on which they are based are hypotheses, merely. 

/ ^ 
Wipi Weiss and Bucuvalas (1980a) , we, are not investigating these and other 

components of frames of reference in concrete settings of practice. 

- N 

THREATS TO USABLE KNOWLEDGE 

In further extending the transactional model we may view threats to know- 
ledge claims as rebuttals (R) to practical and theoretical arguments which 
affirm, explicitly or implicitly, the adequacy, cogency, or releva^e of 
knowledge. In contrast to other approaches which encourage the separate 
exploration of rival hypotheses about causation (e.g., Cook and Campbell, 197 St) , 
or those dealing with ethical norms (e.g., MacRae, 1976) the function of threals 
to knowledge adequacy, relevance, and cogency is to challenge both the subs tan- 
tial and analytic bases of empirical as well as normative claims. Therefore, 
while threats to adequacy, relevance, and cogency provide alternative 
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interpretations of the same data or evidence, they are not limited to assessments 
of the validity of causal inferences. 

Figure 3 summarizes three classes of threats to usable knowledge. The 
majority of thes* threats reflect methodological and practical issues not 
addressed by Campbell and Stanley (1963) and Cook and Campbell (1979) in their 
listing of classes ot threats to the internal, external, and statistical conclu- 
sion validity of causal inferences. Indeed, and as repeatedly noted, these 
validity threats are exclusively oriented towards standards of adequacy ^nd 
relevance which are" causal and designative, respectively. The one exception 
to this exclusive concentration on causal and designative standards is 
"irrelevant responsiveness of measures," a threat to external validity where 
the imperfect validity iof measures in adequately representing experimental 
outcomes valued according to conflicting standards held by diverse stakeholders 
is overcome by multiple operationism and| triangulate ion (Campbell, 1975b:79-80; 
Campbell and Fiske, 1959; Webb et al ., 1966). This threat to validity implies 
an interpretive test which is not easily reconciled with an experimentalist 
platform which contends that outcomes should be independent of the preferences 
of different stakeholders (Campbell, 1979b), 

Threats to the usability of knowledge may be divided into three classes; 
cogency, relevance, and adequacy (Figure 3). Threats to the cogency of knowledge 
claims are of two main types: misjudged cogency and misplaced cogency. Misjudged 
cogency, a topic of standard statistical textbooks, is illustrated by errors of 
practical judgment which occur when one sets statistical confidence limits too 
high (Type I Error) or too low (Type II Error) in testing the null hypothesis. 
By contrast, misplaced cogency occurs when one^ correctly aet9 statistical confi- 
dence limits but addresses the wrong problem. The threat of misplaced cogency 
is evident in John Tukey/s admonition to applied social researchers: "Far better 
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an* approximate answer to the right question, which is often vague, than an exact 
answer to the wrong questions, which can always be made precise 11 (quoted by Ros^, 
A 1977:23). This first-order threat has-been generalized by Kimball (1957), Raiffa 
(1968), and Mitroff (1974) as Error of the third Type (Eju) and discussed by 
Campbell and Stanley (1963:6-7) under the heading of misplaced precision in one- 
shot case studies. 

■* 

Threats po knowledge relevance are also of two main types: misplaced rele- 
vance and untimely relevance. Misplaced relevance involves the projection of 
cogent knowledge claims which are relevant to one kind of purpose when, instead, 
cogent knowledge claims relevant to another kind of purpose should have been 
produced. This second-order threat is noted with great frequency in published 
-literature on policy research (for example, Coleman, 1972; Rein and White, 1976). 
The threat of misplaced relevance is also noted by- proponents of multiat tribute * 
utility analysis who contend that ex^trim^ntal program evaluations offer 
designative claims but not evaluative and advocative ones (Edwards, Guttentag, 
and Snapper, 1975:140). By contrast untimely relevance, a second-order threat 
that is more easily overcome, involves the production of relevant information 
too late to satisfy the needs of one or more stakeholders (see, for example, 
Weiss, 1977). 

Threats to knowledge adequacy are more diverse and complex than those per- 
taining to relevance and cogency. Among the major threats to knowledge adequacy 
are: 

1- Misplaced Adequacy : The use of the less appropriate of two or more 
classes of truth tests when, instead, the more appropriate truth test 
should be employed. This second-order threat is fot!n&~-lQ theoretical 
and practical disputes surrounding the appropriateness of contending 
worldviews, paradigms, and frames of reference for policy research 
(for example, Tribe, 1972; Patton, 1975; Reii^l976; MacRae, 1976). 
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Subjectivity : A classical causal test Is used to explain human behavior 
when, Instead, the explanation should be supplemented or replaced by pne 
founded on subjectively meaningful action. Claims about the effective- 
ness of federally-sponsored social experiments\are frequently challenged 
on grounds of subjective Inadequacy (e.g., Trend, 1978), another second- 
order threat. * 

Ref lexlvlty : A quasi-causal test Is used to affirm the social or his- 
torical necessity of s<4fce process or event when, Instead, such processes 
or events are subject to human reflection, Initiative, and control. 
This second-order threat Is sometimes applied to quasi-causal theories r 
of revolutionary social change. Such theories are challenged on grounds 
that predictions of socio-historical events hold true if and only if 
reflection by stakeholders does not lead them to change their values or 
behavior; or if unpredictable factors that arise through creative re- 
formations of social problems do not intervene (Maclntyre, 1973). 

Mlsclassif ication : This second-order thteat may be invoiced to determine 
whether a typological test results in the placement of events, actions, 
or persons in the wrong class. The creation of social pseudo-problems 
by labelling healthy persons as deviants reflects classif icational 
inadequacies that derive from unexamined paradigms and social myths 
(see, for example,, Lowry, ^.974; Gregg et al . , 1979). 

Misrepresentation : The use of a particular representational test, 
whether statistical or theoretical, when anotheV more representative 
rule should have been employed. The under enumeration of minorities in 
the 1970 U.S. Census illustrates this second-order threat. 

Perspectivity : The use of an analogical test as a literal surrogate 
for some social process when, instead, the analogy is no more than a 
perspective or metaphor of that process. Challenges t6 the adequacy 
of quantitative policy models illustrate this second-order threat to 
metaphorical adequacy (e.g., Strauch, 1976). ^ s 

Objectivity ; The use pf a teleologlcal te^t to explain action when, 
instead, the eiplanariop should be supplemented or replaced by a 
quasi-causal test which identifies the operation of humanly objecti- 
vated but unreflected law-lifce regularities. The concept of unanti- 
cipated social consequences and the "self-fulfilling prophecy" 
illustrate this second-order threat (Merton, 1976). 

Spuriou8ness : Use of a clinical test involving one symptom or set of 
symptoms to diagnose or treat a social ill when, instead, some other 
symptom or set of symptoms is a better indicant of the problem. Know- 
ledge about the diagnosis and treatment q£ mental and physical illnesses 
is often subject to second-order threats of spurious syjnptomatogy. 

Misplaced Comparison : Use of a comparative test to adopt a reform that 
has succeeded elsewhere when conditions surrounding the reform are not 
sufficiently similar to the case at hand. Misplaced comparison is a 
continuous second-order threat to knowledge claims about government- 
sponsored "exemplary' 1 projects. 
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10. Counterauthentication ; Use of a personal, ideological, ethical, or 
methodical test when some other person, doctrine, norm, or procedure 
is more qualified, orthodox, fair, or sanctioned. Knowledge prpdjuced 
by mystics, seers, gurus, scientists, and .expert panels and commissions 
is typically threatened by diverse forms of counterauthentication, as 
is knowledge which originates in ideological doctrines, ethipal systems 
and approved technical conventions of science. 

• 

11* Substantiality : A (necessarily) presumptive ontological claim about 
the n^tute of social reality, r human nature, or knowledge certified 
epist^mological or ethical claims when, instead* such claims should 
be argued on substantial grounds. This second-order threat is evident 
in Campbell's criticisms of the naturalistic fallacy (i.e., deducing 
ethical from non-ethical ^premises) t in contemporary sociobiology 
(Campbell, 1979a) and in efforts of critical social theorists (e.g., 
Habermas, 1975) to challenge presumptive ontological claims of logi- 
cal positivism on grounds that such claims, since they represent 
conclusions 'of substantial arguments, are corrigible and redeemable 
through social discourse. 

12- Misplaced Reflexivity : Claims about the emancipatory role of self- 
ref lection and reasoned discourse are treated as if they refer to 
concrete cont^exts of practice when, instead^ they are unrelated to 
ongoing practices.. Misplaced reflexivity is a standing threat to. 
much work carried out in ethnomethodology , phenomenology, and critical 
£heory. 

These- classes of threats to knowledge adequacy may stand in a complementary 
relation, as when the threat of objective inadequacy induces the use of a com- 
bined teleological and qua^i-causal truth test. In other cases threats to 
knowledge adequacy expose fundamentally Irreconciliable standards of appraisal, 
for example, when subjective inadequacy reveals that nomic connections .(laws) 
appropriate to, knowledge claims in physics are inapplicable to sociocultural 
systems. Finally, threats to knowledge adequacy may be extended in the form 
r of additional classes. The framework described above makes no claim to exhaus- 
tiveness or universality. 

(Place Figure 3 about here) 
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Figure 3 
THREATS TO USABLE KNOWLEDGE 



\ Class/Type 



Representative Threat 



THREATS TO 
COfcENCY 

— Misjudged Cogency 



Misplaced Cogency 



THREATS TO 
RELEVANCE 

— Misplaced Relevance 



— Untimely Relevance 



THREATS TO 
ADEQUACY 

— Misplaced Adequacy 



— Subjectivity 



— Reflexivity 



Misclassif ication 



Misrepresentation 



Setting statistical confidence limits too 
high (Type I Error) or too low (Type II 
^Error) in testing the null hypothesis 

Correctly setting statistical confidence 
limits for the wrong problem (Type 1 III 
-Error) 



Production of cogent knowledge' claims that 
are relevant to the wrong purpose 

Production of cogent and relevant know- 
ledge claims too late 



Use of less appropriate of two or mov€ 
classes of truth tests when, instead, a 
more appropriate truth test should be 
employed 

Use of causal test when explanation should 
be supplemented or replaced with one base<£ 
on subjectively meaningful action 

Use of quasi-causal test without recogni- 
tion that social processes are subject to 
human reflection, initiative, and control 

Use of typological teat results in place- 
ment of events, actions, or persons in 
wrong class 

Use of less appropriate of' two or more 
representational tests, whether statisti- 
cal or theoretical 
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Figure 3 (CONT'D) 



Perspectivity 
Objectivity 



Spuriousness 



Misplaced Comparison 



Count er au thent ica t ion 



Substantiality 



Misplaced Reflexivity 



Use of analogical test as literal surro- 
gate rather than perspective or metaphor 

Q) Use of teleological °test when explanation 
should be supplemented or replaced by 
quasi-causal test which identifies humanly 
objectivated but unreflected law-like regu- 
larities 

Use of clinical test involving a set of 
symptoms which are less appropriate 
diagnostic or treatment indicators than 
another 'set of symptoms 

Use of comparative test when two or more 
cases are noT similar. 

Use of personal, ideological, ethical, or 
methodical test when some other person, 
doctrine, norm, or procedure is more 
qualified, orthodox, fair, or scientifi- 
cally sanctioned 

Use of a (necessarily) presumptive ontolo- 
gical claim to certify epistemologicaf or 
ethical claims when, instead, such claims , 
may be argued on substantial grounds 

Claims about the emancipatory role of self- 
reflection and reasoned discourse are 
treated as if they refer to concrete con- 
texts of practice when, instead, they are 
unrelated to ongoing practices 
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CONCLUSION 



Problems of knowledge production and use cannot beosatisf actorily clarified 
or alleviated by making exaggerated and facile distinctions between professional 
social inquiry and ordinary knowledge; nor should we accept the patently false 
conclusion tha£ knowledge derived from one or the other source is inherently 
superior. The task is rather to distinguish between approaches to knowledge 
creation, and use which recognize the critical function of reason in appraising 
knowledge claims and those which do not. Jhe metaphor of .the experimenting 
society, while it* has introduced reasoned discourse into the examination oF 
causal inferences, fails to recognise that reforms are symbolically mediated 
social processes aiming at changes in the structure and functioning of some 
social system. Accordingly, experimental outcomes * cannot be said to be indepen- 
dent of the preferences of stakeholders in, social reforms. - > 

The success of reforms depends upon rationally motivated consensus that 
some future social state is possible and desirable. Reforms are processes of 
reasoned argument and debate where competing standards for appraising knowledge 
claims include but are not limited to rules- for making valid causal inferences. 
The jurisprudential metaphor not only captures these diverse standards for 
assessing knowledge claims; it also directs attention to processes of knowledge 
creation and use as critical spcial transactions involving issues of the compar- 
ative adequacy, releVance, and cogency of " knowledge claims. ^ 

The jurisprudential metaphor has been extended' and specified in the form 
of a transactional model of argument. The transactional model, since it dis- 
tinguishes between analytic and substantial arguments, is well-suited for 
critical inquiries into competing standards for assessing theoretical and 
practical claims alike. The transactional model provides a visual schema. for 
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mapping arguments, compels a reflective and critical posture towards presupposi- 
tions of knowledge claims, yields a classification of truth, relevance, and 
cogency tests, and permits a provisional listing of classes of threats to . 
usable knowledge. The role of the transactional model is not limited^o retro- 
spective inquiries into standards of knowledge assessment employed by~contendlng 
stakeholders, since an awareness of threats to usable knowledge helps antici- 
pate diverse challenges to knowledge claims. By supplying the contours of a 
critical social science of knowledge applications— that is, a social science 
which uncovers and raises to a level of explicit consciousness those unexamined 
prior assumptions and implicit standards of assessment that shape and distort 
the production and use of knowledge—the transactional model may contribute to 
individual and collective learning capacities and, thus, to emancipatory social 
reforms. 
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ABSTRACT 

4 - 

This paper discusses the concept, frame of reference and 
the Interplay between the Individual's frame of reference, 
his or her location 1n social structure, and perceived 
responsibilities. The components of frames of references 
are examined for knowledge producers and teachers as potential 
knowledge users. Frames of reference are a critical Inter- 
mediary between knowledge and Its use and therefore our 
attention to frames of reference 1s necessary for predicting 
knowledge use. 
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FRAMES OF REFERENCE AND THE PREDICTION OF KNOWLEDGE USE 

By 

Burkart Holzner and Leslie Salmon-Cox 



Introduction; The Concept Frame Of Reference 

* All cognitive activity, whether of the scientific, professional, or commonsense 
variety, establishes meanings by relating an object of inquiry to other objects, to 
standards of judgment, and underlying assumptions thus enabling persons to arrive at 
interpretations that appear to them sensible and grounded. The concept "frame of 
reference" denotes a structure of those underlying assumptions, standards and 
interests that provide the framework to* which specific interpretations are related 

» i 

and within which they make sense. 

/ - ence is rather obvious when we make 

judgments, for example about temperature. The weather forecast that would call a 
day in July "cold" in the Northern hemisphere would surely call the same 
temperature "warm rt if it happened in January. Similarly obvious is the role of 
spatial frames of reference in our orientation to what is up or down, left or right, 
forward or backward. We are familiar with frames of reference in political judgment 
when we classify opinion as right of left, conservative or progressive. 

The term "frame of reference," while gaining somewhat broader currency 
recently, tradit^ally has been used both in methodology to" describe the status of 
certain assertions, and in social science, where it has been used^to^describe 
explain certain cognitive phenomena. For example, E. W. Hobson (1923) distinguished 
between two types of concepts in science, those that have direct observational 1 
counterparts, and those that do not. These latter concepts, Hobson says, "are formed 
' - i - ' o ' ' ,? - 
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by an effort of constructive imagination for the purposes of the representative 
scheme" (Hobson, 1923:23). They thus function; as components of a frame of 
reference. Such frames as a whole cannot be inferred from experience but are 
necessary to make meaningful observations. jThe same use was made of the term by 
. sociologists like Howard Becker, and Talcott Parsons and Edward Shils. For example, 
Parsons and^Shils state "the frame of reference of the theory of action involves 
adtors, a situation of action, and the orientation of the actor to the situation" 
(Parsons and Shils, 1952:56). This conceptual framework is proposed as a necessary 
set of a prioris for social science. In this meaning the term. is used to denote a set of 
assumptions necessary to determine the subject matter to be studied, and the 
orientation to such study. 

In ea^Tcognitive psychology the concept was elaborated by Muzafer Sherif 
(1936:33). "When we say 'up,' we mean 'up' in relation to something that is below. 
When we say 'far- we" mean -far' in relation to something near". The properties of 
perceived phenomena are then assessed by the subject in relation to some standard, 
they are not simply results of the stimulus.! tself . Sherif and Cantril.thah defined the 
concept frame of reference as follows: 

The term "frame of reference" is simply used to denote the functionally related 
factors (present and past) which operate at the moment to determine the 
particular properties of a psychological phenomenon (such as perception 
judgment, affectivity) (Sherif and Cantril, 1947:34). perception, 

S 

It is interesting to note that the frame of reference phenomenon, namely the 
fact that respondents interpret stimuli in terms of their own definitions of the 
situation rather than that of an investigator, was perceived as a nuisance in social 
research. For example; Charles F. Cannel and Robert L. Kahn in their chapter 6n 
interviewing in the Handbook of Social Psychology (1968) deal with the frame of 
reference problem as it impinges on inquiry. 

In formulating questions the research worker has only three possibilities with 
^r«»pect to the frame-of-reference problem. He can ignore it, he can attempt to 
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They indica|e that to ascertlin the subject's frame of reference, one needs to probe 
into the reasons for respondents' answers. And then they continue to say 

each respondent is usinir thP rAcooJ^lI 7 ceriain tne fram e of reference 

They make several practical recommendations as to how this might be done. 
Clearly, this treatment of the subject deals with reference frames as troublesome 
problems in measurement. 

This attitude sterns to have continued also in much of the work on knowledge 
utUization until recently. Product oriented or physicalist theories of knowledge 
utilization tend tf be relatively insensitive to the frame of reference phenomenon. - 
In this paper we are taking the opposite position. Since knowledge utilization is 
always cognitive activity of some kind, it always occurs within the actors' frames of 
reference. Furthermore, such reference frames are ascertainable and their structure 
and texture can be both described and explained. We see such frames of reference 
from the sociologial perspective as anchored in patterned social organization and 
based upon the cognitive requirements of socially mandated role responsibilities. 

The frameworks of assumptions ahd direction-giving cognitive interests within ' 
which meaning structures emerge is certainly a complex phenomenon when dealing 
with intellectual activity and professional work. And yet, the frames of reference of 
complex fields of inquiry such as scholarly disciplines and their various branches and 
twigs, as well as- the practicing professions and . occupations, can certily be 

472 



ERIC 



descried and analyzed. Much of an nominating nature has been written about "the 
sociological perspective ■ char.eterizing in broad and general terms the of 
reference of an entire diseipline within whieh there are many competing perspectives 
and zehoola, and the same can be said for psychology, or biology, or' the like. 

However, in order t0 understand the way in which we propose to use the 
eoncept frame of reference, we must broaden our purview and draw on the basic 
sociological notion ,h. t for every specific social role one can describe a reared, or 
encouraged, perspective which, so to speak, constitutes the terms of reference and 
perspective for tha, roie. In the broadest sense, then, we mean by frame of 
reference '.he structure of orientation of actors tha, ia anchored in their 
responsibilities and commitments and involves, significantly, mandate or preferred 
cognitive resources as well as standards of judgment. 

We are thus using a concept that is considerably broader than Thomas KuluVs 
concept "paradigm" (Kum,, 1962; 1970, Suppes, ,974). Paradigms order the structure 
of Pledge u, a field of inquiry in relation to exemplary feats of research and 
. theory. Multiple paradigms m.y^ell occur within one institutionalized frame of 
reference, but paradigm shifts may indeed alter such frames. Further, the idea of 
paradigms emphasizes ^te strictly the structure of intellectual activity and the 
resuhingknowledge structures aa such, and i, is useful in this context. But b, our 
relating frames of reference ,o social roles and thus to institutions and 
organizations, we mean ,o emphasize their social anchorage and the. social 
, organization of cognitive activity. We are doing this, asthe body o, this paper will 
make clear, because of the significant insights into the uses of knowlcW that can be 
gained in this manner. ^ |/ 

At this point we must foresail a miainterpretatiol which for some readers w.Tl 
be do* at hand. ,t ia the notion that.the use of concept Uke frame of reference 
and paradigm requires one to ,d<ypt a rela.ivis.ic epiatemology, or a sociological 



reductiortism. Some writers seem to have concluded from the empirical fact that 
inquirers in different communities proceed in terms of often drastically divergent 
frames of reference as well as paradigms, that the resulting claims to knowledge can 
only be judged in relation to the standards and tifeliefs in terms of which they were 
originally offered. Ultimately, this may lead to the belief that the validity of 
knowledge claims can only be established by creating#-in whatever manner— social 
consensuses that they are correct. In some arguments, standards of truth are thus 

simply reduced to impositions of social power. These conclusions seem to us not only 

V 

unnecessary but also false. 

Ndt only specific substantive knowledge claims can be critically scrutinized 
(and this always occurs within the structure of some frame of reference), but frames 
of reference themselves, and especially the standards of judgment they encompass, 
can and have been the subject qf critique. History off ers little support for the view 
that all knowledge claims and all frames of reference were of equal mention the 
contrary there has occurred and is occurring now a process of critique on the basis of 
increasing!^ universal standards of judgment. Rational inquiry has led to hard and 
compelling conclusions which remain valid despite the disagreement <jf multitudes. 
Indeed, we consider the investigation of frames of reference to be one aspect of the 
critical extension of rationality, not an intrusion of irrationality. 
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The Dimensions of Frames of Reference 

In an empirical inquiry into the dimensions^ reference frames, Holzner, 
Mitroff, and Fisher (1976) studied in detail the frames of reference of several 
scholars who had conducted significant social inquiries in law, psychiatry, 
psychology, and sociology.* In this inquiry a set of nine categories emerged which, 
taken together, permit the comprehensive description of frames of reference. We 
will use these categories here in order to demonstrate the multidimensional structure 
of frames of reference. The categories are: 

1. Commitments and Responsibilities: 

Role responsibilities and commitments are certainly the grounding of frames ot% 
reference that provide moral as well as motivational foundations. These 
commitments ofteij constitute a structure which serves as a basife code for inquiry 
that provides generative rationales for the direction and conduct of cognitive 
activity. This code, depending on the nature of responsibilities, can be described 
with regard to differentiation, complexity, unity, and organization. Clearly, certain 
occupational and professional roles require that highly divergent situations be dealt 
with that are difficult to integrate. Other roles, by contrast, may deal with 
relatively unitary sets of responsibilities. 

2. Problem Selection and Formatibn 

The jnanner in which problems are detected and formed into questions so that a 
solution clan be pursued differs across frames of reference, largely as a function of 
the conception of what an acceptable solution to the problem can possibly be. For 
example, the solution of a research problem in psychology may not at all appear to a 



•Note: A report on this work is available in the knowledge use working paper "An 
Empirical Investigation of Frames of Reference" (Holzner, Mitroff and Fisher,* 1976) 
which summarizes the frames of reference study of 1975-1976." 
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teacher as having anything to do with the solution of a concrete pedagogical problem 
in the classroom. This dimension of frames of reference is particularly important to 
ill-structured problems. * 
3. Strategies of Problem Solution 

For each frame of reference, one can describe a set of preferred modes for 
solving problems. This, of course, will depend on the available cognitive resources 
and especially involves the setting of intermediary goals in relation to an ultimate 
goal. 

•4. Rules of Inquiry and Analtyical Devices 

This category is used to describe the inquirer from the point of view of his 
subscription to certain rules of inquiry which are" considered valid by him and his 
preference for or competency in the use of certain methodological and analytical 
devices.. Professions both of the consulting and practicing variety as well as 
disciplines go to great lengths in attempting to set standards for rules of inquiry and 
methodologies. 
, 5. Conceptions of the Nature of Facts 

Conceptions of "faeticity come in a variety of forms. Facts may be rank 
ordered by an inquirer According to their significance or triviality, or they may be 
distinguished as relatively soft versus relatively hard facts and the like. This has 
much to do with the trustworthiness of knowledge and the core realities to be dealt 
with in terms of the frame of reference. 
6. Truth Tests 

k. 

Truth tests are validations of the basic beginning points as well as knowledge 
outcomes in terms of critical, socially and culturally structured occasions. Truth 
tests themselves are normally believed to require no further validation. It is through 



♦See for example Mitroff, Bets, Pondy, and Segasti (1974). 
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the connection of knowledge claims with these occasions for critical examination 
that knowledge acquires the somatization of representing reality. Truth tests can 
- be directed at issues of relevance, cogency, and knowledge adequacy. ' \^ 

Truth tests range over a wide spectrum from the empirical and rational tests of 
science to mystical tests to which "inner experience" may be subjected. In a 
practical profession such as teaching a variety of pragmatic truth tests are likely to 
play a major role. 

7. Delimiting the Domain of Inquiry: The Nature of Boundaries 

Frames of reference orient one to a variously delimited "field." The boundaries 

i 

may be drawn with great precision and finality, or may be very diffused. The 
category includes as well conceptions of alternative frames 0 f reference and rules 
for bridging framework boundaries as well as maintaining them. 

8. Articulation and Codification of the Frame of Reference 

A frame of reference may be symbolically articulated as a structured. cultural 
object, and/or it may be codified into interpersonally shared, institutionalized, rule 
systems. There is, of course, a difference between articulation and codification. A 
codified frame of reference is necessarily an institutionalized one. It is this aspect 
of codification that has a particular impact on patterns of knowledge use. 
Codification includes such things as formal training curricula, examinations and 
certifications, as well as particular requirements for the way in which one deals with . 
knowledge. Certainly, very, idiosyncratic frames of reference can be articulated and 
they have been. Inarticulated, implicit reference frames can often be found, for 
example, in the taken-for-granted commonsense of a community. Frames *of 
reference of teachers are usually of the implicit rather than articulated and codified 
variety. In this regard they are in sharp contrast to the frames of j-eference^of 
researchers who are required to make their assumptions, and methodologies as 
explicit as they can manage. 
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9. Reflectivity £ 

Reflectivity refers to the ability of a^bje^to^view himself as an object. 
Frames of reference differ very sharply in the degree of reflectivity. In general, the 
more reflective frames we would expect to be more effective in adequately assessing 
and critiquing other frames. 

This listing of categories for the description of frames of reference emerged' 
from the close-up, empirical study of scholarly reference frames which we cited 
above (Holzner, Mitroff, Fisher, 1976). It is therefore somewhat slanted in the 
direction of articulated, codified, reflective, and very complex reference frames. 
However, we do feel that this inventory may have a certain practical utility in 
directing attention to critically important aspects of reference frames which need to 
be taken into account as one thinks about knowledge utilization plans. It is; for 
example, unlikely that knowledge stUes not clearly linked to responsibilities will be 
perceived as relevant for use. Or, knowledge claims proffered in a style violating 
pref erred knowledge adequacy standards are likely to be rejected. 

But we must increase the complexity of our analysis somewhat in-order to be 
adequate to our task. Frames of reference are not only individual attributes, they 
occur in the context of social organization and structure. We will now turn first to 
the task of reviewing the social organization of a frame of reference in its 
institutionalized form, and then turn to the location of frames of Reference in the 
knowledge system of a society. 




378 



10 

The Social Organization of Frames of Reference 

Frames of reference can be described at the individual, the collective, the 
organizational, and the institutional levels. Any individual could be described in 
terms of the set of frames of reference available for structuring cognitive activity. 
Frames of reference can be said to occur at the collective level to the extent that 
they are shared across sets of individuals. However, such sameness of perspective 
does not necesarily imply institutionalization or structure. The organizational level 
is of particular interest to us as frames of reference can be said to be characteristic 
of organizations (Blau and McKinley, 1979) and because codified frames of reference 
themselves require structured organizations for their operation and maintenance. In 
the case of the professions such structured organizations have became fully 
institutionalized and their frames of reference are thus embedded in socially 
sanctioned and often legally-anchored institutions. 

Since our task is to exlore the predictive utility of the concept frame of 
reference for knowledge utilization especially in institutional and organizational 
settings, we will deemphasize the individual or personal level. This, of course, should 

• not be taken to mean that individual variation, based upon socialization, formal 
education, life histories as well as personality characteristics, is unimportant-on the 

' contrary. The world views and perspectives of leadVs have, for example, for good 

reason, received enormous attention in the scholarly literature. Innovators and 

I 

entrepreneurs have often played major roles in organizational change and in the 
"** development and adoption of innovations. However, it is at least equally appropriate 

to focus in the context of this paper at the level of social organization v / 

. To help us with a first orientation to the matter, let us take a very brief look ^ 
at an area in which the institutionalization as well as social organization of frames 
of reference is particularly apparent; the institutionalized forms of intellectual 
activity. From this we will turn to^he institutinalization of frames of reference in 
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practicing professions. What we have in . mind here is analyzing certain 



organizational structures from the point of view of their serving as social 

frameworks for knowledge. Edward Shils gives a very useful and precise explication 

of w^at this notion involves: 

Institutions create a resonant and echoing intellectual environment. . .by 
institutionalization of an intelletual activity I mean the relatively dense 
interaction of persons who perform that activity. The interaction has a 
structure: the more intense the interaction, the more its structure makes place 
for authority which makes decisions regarding assessment, admission, promotion, 
allocation. The high* degree of institutionalization of an intellectual activity 
entails its teaching and investigation within a regulated, scheduled and 
systematically administered organization. The organization regulates access 
through a scrutiny of qualification, provides for organized assessment of 
performance, and allocates facilities, opportunities, and rewards for 
performance— for example, study, teaching, investigation, publication, 
appointment, and so forth. It also entails the organized support of the activity 
from outside the particular institution and the reception or use of the results of 
the activity beyond the boundaries of the institution. An intellectual activity 
need not be equally institutionalized in all these respects. It should be 
remembered that an intellectual activity can be carried out fruitfully with only 
a rudimentary degree of institutionalization (Shils, 1970). 

r 

A specialist in any field, be it nuc clear physics of the study of classics, 
educational research or sociology, is -carefully educated to internalize a frame of 
reference that emphasizes his or her core responsibilities, commitments, truth tests, 



etc. This frame of reference is clearly interactive and interdependent with the 
organizational structures which Shils points to in his description of institutionalized 
intellectual activity. Thus, adherence to the frame of reference becomes an official 
requirement,, departures or innovations needing to be justified by elaborate 
argument. Graduate education demands that students master an appropriate manner 
of forming problems, and that they be aware of the problem priorities as they are 
pursued in their discipline. Strategies of problem solution, methodologies, and 
standards as to the nature of fact are hardly matters of great permissiveness. At^the 
same time, it it the case that they differ widely acrbss disciplines and professions as 
much as the specific nature of reality or truth tests. 
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We draw from these facts the >W^«^eral conclusion that institutionalized 
frames of reference are embeddecf ih strtictured social frameworks for knowledge. 
The idea that groups, solidarities, organizations, or communities are alio frameworks 
of knowledge has first been formulated by Gurvitch in his work The Social 
Frameworks of Knowledqfe (1972). He examines in this work how l&cal groups develop 
&Jir own modes of knowing. In another illustration he explores in detail how forms 



of knowledge are Stabedded in the social organization of a factory. The important 
point is that such social organization^ patterns are linked to the way in which tasks 
are organized, knowledge resources stored, so that cognitive activity can function 
within the structure of the institutionalized frames of reference. 

The practicing and consulting profession can be considered elaborate 

frameworks for knowledge. The profession of medicine provides an example of a 

/ 

particularly highly jrtructured social framework for knowledge, whereas such 

professions as social work, education, counselling are much less structured by 

comparison. ^\\e*e social frameworks for knowledge, in the case of the practicing 

professions, are necessarily organized around the central practical responsibilities of 

the profession and the core social performances and identities with which it deals. 

This point is well made by Freidson about the profession of medicine: 

Within its own institutions, protected by its organized autonomy, the profession 
has developed knowledge of its own, and by virtue of being a consulting , 
profession, a capacity of its own to shape the behavior and experience of the 
layman independently of the lay community. In those institutions the profession 
ddes not 'merely treat a biological state by biochemical or physical technique: it 
also organizes the social identity of the layman into being a patient. Thus, in 
applying its knowledge, the profession cannot avoid making social as well as 
"purely medical" decisions about the people it depls with . . . when an occupation 
\ arises to serve some need or demand on the part of the lay community, and 

subsequently succeeds in becoming a profession, it gaihs the autonomy to 
become at least in part self-sustaining, equipped to turn back and shape, even 
create that need anew, defining, selecting, and organizing the way it is 
expressed in social life (Freidson,1970). 

The analysis of frames of reference as predictive tools for knowledge 
utilization requires attention to the struture of the frames of reference themselves 
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but also to the social frameworks for knowledge in which they are embedded. This 
point goes evey beyond the argument offered by Nathan Caplan (1979) in the two- 
communities theory of knowledge utilization, which deals essentially with the gap 
between social scientists as knowledge producers and policy makers. Within a 
somewhat different conceptual framework an approach highly sensitive to the points 
about the nature of social organization and frames of reference has been developed 
by Paul Hood (1976). In this study data are collected on the characteristics and 
information needs as well as information dissemination patterns through survey 
instruments and field interviews with persons who represented a great variety of 
educational roles as well as different locations.* 

Having given consideration to the issues of social organization surrounding 
particular frames of reference, we now must tutn to the more encompassing context 
in the societal knowledge system within which the structures we 4iave discussed this 
far have their location. 
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See also Hood and Cates, 1978. 
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Frames of Reference in the Knowledge System 

While in the previous section we have dealt with the social organization of 
frames of reference in the sense of asking the question what social arrangements, 
resources, controls, and the like sustain a given framework and perspective, we must 
also ask the question of the distribution and location of the socially organized 
frameworks within society itself. For this purpose the concept of the social 
knowledge system is useful. 

This conception is related to the already cited work of George Gurvitch (1972) 
on social frameworks of knowledge, but it was even more clearly articulated in the 
studies of the economist Fritz Machlup. Especially important was his book The 
Production and Distribution of Knowledge in the United States (1962) and the more 
recent work Knowledge: Its Creation, Distribution, and Economic Significance. Vol. 
I. Knowledge and Knowledge Production (1980) and Vol. n. The Branches of Learning 
(1982). Machlup, as the title of the first book indicates, concerns himself with the 
role of knowledge in the economic system, its production and distribution. This was 
at the time a novel conception which caused many shocked misunderstandings since 
knowledge was treated like other economic goods. 

Holzner and Marx (1979) developed the sociological notion of the knowledge 
system in society by drawing attention to the fait that certain analytically distinct 
aspects of social systems come into focus when one attends to the structured 
distribution of knowledge related activities, institutions, organizations, roles, 
persons, and resources. Thus^the conception of the social knowledge system is akin 
to that of other analytical concepts focusing on specific aspects of social systems 
such as the power structure, or the communications network, or the Economic 
system, and others. Obviously, each of these modes of abstracting from the complex 
reality of society is interdependent with all the others. Surely, there are 
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Relationships between social systems of pow^r and of knowledge. However, it is 
fruitful for the task at hand to focus on the knowledge system itself. 

Holzner and Marx used five knowledge functions to differentiate components of 
the knowledge system, namely knowledge production, the organization of knowledge 
into coherent bodies, knowledge storage, knowledge distribution and accessing, and 
knowledge use. Intersecting these functional differentiations are major institutional 
domains such as education, health care, social policy, and innumerable others. 
Further, distinctions needs to be made concerning the relative centrality and 
peripherally of these domains. For example, while great efforts have been made in 
recent years to increase the scope and quality of educational knowledge production, 
it can hardly be claimed that the educational profession, especially those dealing 
with elementary education and their social frameworks for knowledge, occupy a 
highly central place in the American knowledge system. . 

If we look at the flows of knowledge through the system to the various points 
at which knowledge is put to use, that aspect especially of institutionalized frames 
of reference that we have called truth or reality tests becomes particularly 
significant. It serves in part as a set of filters impinging on the channeling of 
knowledge flows and directing them. That is, from the point of view of each frame 
of reference, knowledge claims are critically examined as to their relevance, 
adequacy, and cogency. Relatively few studies are available that deal explicitly and 
empirically with the nature of truth tests. An important exception is the work of 
Carol H. Weiss and Michael Bucuvalas, (1980) who have examined the framed of 
reference of executives in certain- administrative branches of the federal 
government. 

Keeping in mind the strategic focus on the socially structured distribution of 
truth tests in relation to frames of reference and social frameworks for knowledge, 
we now turn to certain observations about frames of reference in education. 
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Observations on Frames of Reference in Education 

In recent decades deliberate efforts have been made to improve the scientific 

knowledge base for educational policy and practice. New research and linkage 

organizations have been established and professional associations have .been 

profoundly affected by this movement. Similar efforts have been made in other 

fields, such as health care, and mental health. The underlying belief is that policy 

and practice could be dramatically improved by increased reliance on high quality 

scientific research and by considering policy changes as experimental 

interpretations. This belief is, well formulated by Charles A. Kiesler (1981), who 

writes not about education but about mental health the following: 

The potential for an effective national policy is clearly there. However, riven 
various uncertainties in the knowledge base, the most dependable policy 
strategies would rest on feedback loops with evaluation and policy research In 
this view, mental health policy should be well articulated, experimental in tone, 
and carefully oriented towards assessment of the outcomes of potential 
treatments and their cost effectiveness and cost benefit. If we could all agree 
that is indeed the wisest approach to the development of mental health poUcv 
then we can also all agree that the current mental health policy is a failure 
since it is none of those things. 

This stance is closely related to the philosophy and methodology formulated by Don 
Campbell around the notion of "policy as experiment" 0971, 1975, 1982; Dunn, 1982). 

In part the science oriented reform movement has deliberately attempted to 
alter both frames of reference and social frameworks for knowledge in educaology 
formulated by Don Campbell around the notion of "policy as experiment" (1971). 

In part the science oriented reform movement has deliberately attempted to 
alter both frames of reference and social frameworks for knowledge in education. In 
fact, in a few instances action was taken apparently based deliberately on the notion 
that planful intervention in frames of reference and social frameworks for knowledge 
would lead both to improved knowledge production and use. An important example is 
reported by Carolyn Hodges Persell in a paper she presented at the meetings of the 
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American Sociological Association in New Orleans in 1972, "The Utilization of 
Sociological Ideas in Organizational Planning, A Case Study."* In the early and mid- 
sixties the leadership of the American Educational Research Association, especially 
Richard Dershimer, who became executive officer of AERA in 1964, and John 
Goodlad, who was the then incoming president of AERA, became convinced that the 
quality of educational research and the centrality of educational research in the 
American knowledge system could both be improved by applying to the structure and 
policy formation of AERA insights gained from the sociology of science. An effort 
/ was made to cull from the body of empirical evidence and theory of the then-existing 
f sociology of science those insights that could be translated into institutions and 
practices within AERA. While the concept "frame of reference" did not play a major 
role, ideas concerning standards of judgment, methods of evaluation, incentives and 
rewards, and the structuring of social frameworks for knowledge certainly did. 

Dershimer had become thoroughly familiar with the sociology of science as it 
was formulated, for example by Warren O. Hagstrom (1965) or Bernard Barber and 
Walter Hirsch (1962). Dershimer was also convinced that the relative lack of 
influence of the educational research community could be addressed on the basis of 
experience gained in natural science policy and by learning from the practices of the 
established natural science communities. 

Persell reports that these concerns were translated into a plan of action late in 
1966. At that time Richard Dershimer and John Goodlad, the incoming AERA 
president, discussed these matters and formulated a plan for an AERA colloquium. 
This idea fell on fruitful soil because AERA at that time was already engaged in 
long-range planning and had created a committee under the chairmanship of Ralph 
Tyler which had recommended that AERA separate itself from the National 
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Education Association in order to become more focused on scientific education 



After a lengthy period of preparation, the colloquium actually occurred in 
1968. Five m^jor papers were presented; 

William D. Garvey, Carnot Nelson and Nan Lin; "A Preliminary Description of 
Scientific Information Exchange irt Educational Research" — an empirical 
investigation of communication processes within AERA, whibh they compared 
with eight other professional associations. 

William Paisley: "The Role of Invisible Colleges" — this was a conceptual paper 
linking the concept of invisible colleges to a series of policy issues in 



Ronald G. Corwirl and Maynard Seider; "Patterns of Educational Research; 
Reflections on Sofoe General Issued — which was based on a series of lengthy 
interviews with sociologists of science. 

Warren O. Hagstrom; Educational Researchers, Social Scientists, and School 
Professionals 19 

Norman W. Storer: "The Organization and Differentiation of the Scientific 
Community; Basic Disciplines, Applied Research, and Conjunctive Domains; — 
This paper presents an overview of sociology of science findings and discussed 
what the author considered most appropriate and effective modes for connecting 
basic and applied interest. 

Certain concepts, namely the notions of professional reward systems, 
communication systems, and their linkage to the quality of research as well as to the 
influence of the profession were utilized by AERA in developing its further policies. 
The resulting changes in the format of the annual meeting, in the introduction of 
recognition for research, and in the communications system were clearly a self- 
conscious attempt to inculcate into one professional organization a scientific and 
science organizational frame of reference built on the then-existing body of 
knowledge in the sociology of science. 

Pef sell's case study provides good evidence that the impact of this deliberate, 



social science based effort to change both frames of reference and social 
frameworks for knowledge in the direction of greater scientific rigor was profound 
and far-reaching. It is interesting to note that the current leadership of AERA, 
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working within an institutional framework that is at least in part the outgrowth of 
the events and planful action of the sixties, is hardly aware of the episode and of the 
role of the sociology of science in the history <Jf their organization.* We may submit 
that this might be taken as an indication of successful institutionalization. 

However this may be, the introduction of scientific knowledge bases into the 
education professions certainly is not unproblematic. While the frames of reference 
of educational researchers have been profoundly shaped by the intellectual 
developments in the field, the establishment of high quality research and 
development centers and laboratories, as well as the work of AERA that we refer to, 
very little of a similar nature has occurred in the profession of teaching, especially 
elementary school teaching. In our view, practical progress in effective knowledge 
utilization would require not only a comprehensive mapping of the frames of 
reference of the various educational professions, but also efforts at frame of 
reference critique and change and responses by schools of education and 

universities. Some part of these points are now being vigorously debated by deans of 

* 

schools of education following an initiative by Hendrik Gideonse (1982). 

We can offer in this paper, for the purpose of a concrete anchoring of our 
discussion, only sketches of the frames of S^ference of two particular role sets: 
elementary school techers and educational researchers. These sets are chosen 
because these frames of reference and social frameworks in which they are anchored 
are strategically significant for all educational knowledge utilization efforts and are 
linked by a great variety of channels. The discussion will concentrate in more detail 
on teachers than on researchers. This is so because much more is already known 
about the frames of reference of researchers and it is the classroom teacher who is 
the responsible professional on whom change strategies focus. 



• Based on personal conversations in 1982. 
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The working reality of teachers is perhaps best explored by Lortie (1975). Two 
more recent examinations, Huberman 0981) and Furth, et al. 0982) focus more 
particularly on those aspects of the teaching role which are of practical importance 
in considerations for improved knowledge utilization. Our own work (Leslie Salmon- 
Cox 1981) on teachers' uses of test information contributes particularly to an 
understanding of both now teachers conceive of their responsibilities and also the 
strategies they employ in assessing their own goal realisation. Eten more recent 
work on teachers and test use has served to confirm some of our findings (Beck and 
Stetz, 1980; Cooper and Leiter, -1980; Miehls ^amTMeehan, 1982; Rice and Higgins, 
1982). 

Some more general state ments^are necessary before we can turn to the specific 
consideration of the components of teachers' reference frames. First, it should be 
remembered that teachers are practicing professionals who are provided only a 
relatively low degree of formal training by comparison to other professionals such as 
researchers, university professors, lawyers, or physicians. Most teachers enter the 
profession with a first college degree, many acquire later a second degree (but rarely 
a third one such as the doctor of' education). This degree is normally received at the 
completion of what is universally agreed to be one of the least rigorous courses of 
study available within American higher education. (Sieber and Lazarsfeld, 1966; 
Gideonse, 1982). As teachers often point out, their "trade" is learned "on the job." 
This has direct and important consequences for the structure and especially the 
degree of articulation and codification of the profession's frame of reference. Much 
greater emphasis goes to implicitly learned practices and skills than to formally 
inculcated rules and rigorously structured theories. Thus, teaching is necessarily 
primarily a practical art. 

Once assigned to a classroom, teachers work in an environment which is 
relatively isolated and private. Further, for a very substantial part of the time 
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interaction with other adults is absent and the teacher exercises tremendous 
authority within the classroom. Even in large school districts with numerous 
ancillary personnel such as counselors, social workers, instructional supervisors, and 
others, moat of a teacher's performances are unobserved and not evaluated by 
professional colleagues. 

Rewards for teaching take two forms, external and internal. Internal rewards 
are of predominant importance. The externally provided rewards such as salary and 
status recognition acts are hardly of paramount attractiveness. Salary is referenced 
to years of service, and sometimes also to attainment of higher educational credits 
beyond the ^ntry level requisites. Status recognition acts are of various kinds 
including, for example, assignment to special ^comtfuttees (frequently with the 
provision of released time from teaching and/or extra money), the award of funds for 
special instructional materials, etc. Teachers themselves usually cite internal 
rewards as being the most important ones. Even in the current time of sharply 
higher teacher salaries (when compared to those of past decades) teaching remains a 
relatively low income profession, reflecting undoubtedly societal appraisal of the 
profession as well. 

Teachers who stay in the profession speak of the satisfaction they receive from 
the conduct of their work itself. They are rewarded by their experience that 
children— under their care and autt^rity and through their work— have changed in a 
positive direction over the period of time the teacher has spent with them. This 
statement is purposively general, not specific to cognitive instruction, because 
teachers' goals are likewise not so limited. 

Elementary school teachers do consider it their responsibility to help children 
develop both cognitive and social skills, with the latter category very broadly 
defined. We have found this orientation in a number of schools, regardless of the 
socioeconomic status v pf the community being served. Teachers talk with equal 
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emphasis of helping children to learn to read and to share, to master arithmetic and 
citizenship skills, to develop self-confidence, independence, self-worth, along with 
elementary cognitive competencies. In relation to these points one needs to 
remember that many teachers, especially those in the primary grades, work in "self- 
contained" classrooms, overseeing a group of children for the entire day and being 

responsible for all that is transmitted during that day. This time/space dimension 

/ 

within which they work supports the holistic nature of their goal structure and it 
facilitates it. 

As teachers' goals and responsibilities are broadly defined, the criteria they 
bring to hear in assessing goal realization must be multifaceted. We have found that 
test data, for example, are useful for only a small segment of performances teachers 
wish to assess. Consonant with the breadth and nature of their goals is the fact that 
much teacher assessment is based on observation of students and direct interaction 
with them. 

In order to discuss this in more detail, let us consider now the components of 
frames of reference outlined in the earlier section of this paper as they apply to the 
frame of reference of elementary school teachers. 

(1.) Frames of reference are anchored in commitment and responsibilities. In 
the highly trained professions we can speak here of "a reference to the "code of 
inquiry," but we need to modify this in the case at hand. Elementary teachers are 
not so much inquirers as they are problem-solvers and their frame of reference is 
anchored in their definition of the scope of their responsibility. As noted above, this 
scope is usually broadly defined. Thus the anchor point of the frame of reference 
becomes commitment to teaching "the whole child" not just a specific subject 
^ma^ter. 

(2.) Problem selection and formation: Teachers have very little power of 
situation control and thus have almost no latitude in the selection of problems to be 
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solved, but they do exercise discretion over problem-solving emphases. That is, 
teachers are presented with a group of students, must accomplish a "year's worth of 
work," and do so with material tools such as textbooks, workbooks, tests, etc., which 
are usually not of their own choosing. But because they define their responsibilities 
so broadly, they do, in fact, shape the problem emphases to be solved. To the extent 
that they emphasize strongly social skitfs, for example, they include an area of 
accomplishment which requires of them expertise beyond that normally provided in 
materials form. 

(3.) Strategies of problem solution; Teachers use a variety of techniques to 
instruct students ranging from the didactic lecture to the physical modelling of skills 
or behavior students are to acquire. Elementary teachers rely less heavily on 
. techniques that necessitate students having long attention spans or sophisticated 

t • t 

cognitive skills. An implication of this is also that the teacher is spending many 
hour* a day isolated from other adults and engaged in forms of interaction quite 
dif fereht from standard interaction among adults. 

(4.) Rules of inquiry; The cognitive style and indeed codes brought to practice 
in the classroom tend to be highly varied and personalized. Relatively little 
technical standardization can be observed which may in part be a functiwToftf 
relatively low degree of formal training, but also a result of the need to manage an 
entire classroom as a continuous and personal performance. 

(5.) Conceptions of the nature of facts: Because of their holistic orientation 
to the scope of their responsibility, visra-vis children, teachers take into account a 
large number of pieces of information about any given child. In this way, their 
conceptions of "facticity" are very broad. They often find equally compelling home 
background factors and notions of "inborn ability," as well as in a much more limited, 
way formal measurements. I 
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(6.) Truth tests: The truth tests teachers employ rely heavily on pragmatic 
validation of ideas within the structure of their actual work. The thing to be 
assessed can range from a textbook to a teaching technique to a student reward 
system to a disciplinary procedure. New procedures or materials are probably best 
^ legitimated for teachers if it is possible to demonstrate that they have "workedMn a 
similar setting already, that is in a setting other than a laboratory or a special 
school. Within their own domain, teachers are inclined to most often employ 
techniques of their own making, such as teacher-made tests, observations, 
interactions to assess the educational outcomes. 

(7.) Delimiting boundaries: Given the nature of basic responsibilities and 
commitments, the boundary lines that teachers draw are quite broad. Again, 
boundaries tend to be drawn in relation to practice. Time delimitations become 
particularly important-such as so many hours per day and per year of teaching. 
Conceptions of other frames of reference and relations to them tend to be of a 
diffuse nature. 

(8. and 9.): Articulation, codification, and reflectivity: Teacher's reference 
frames are relatively low in the degrees of articulation and codification-an aspect 
of frame of reference structure that follows from the core resonsibilities. The 
emphasis on the craft orientation, the need to work in relatively private situations to 
achieve only broadly defined goals with pragmatically and experienti^ly acquired 
skill hardly provide incentives for formal articulation and codification of the frame 
of reference. With regard to reflectivity, however, wide variation can be observed 
especially among those teachers who act as reform or change agents within their 
school. 

This characterization of the frames of references of teachers to us has 
obviously predictive consequences for knowledge use. Items of knowledge, skills* 
techniques, must first enter into the frame of reference and into the domain of 
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practice of teaching on terms compatible with its structure. We now turn to the 
consideration of the frames of reference of educational researchers. 

This set of professionals is highly diverse. As we have described above, it is a 
domain in which deliberate efforts have been'^made to institutionalize frames of 
reference for the purpose of increasing the quality of scientific work, productivity! 
and communication. Because of the diversity within the educational research 
profession, we shall have to discuss it at the level of generalization that subsumes its 
many varieties. Yet, even at this level it is possible to sketch a compelling contrast 
between researchers'frames of reference and those of teachers. The comparison is 
especially useful because the significant differences between the two frame of 
reference s§,ts have often been the iinreflected-upon cause of difficulties in 
eduational efforts at knowledge use. In recent decades there has, however, been 
measurable progress in developing understanding of these difficulties. This is 
evidenced by both the creation of linkage organizations (Salmon-Cox, 1980, 1981; K. S. 
Louis, 1981; Lehming and Kane (eds.), 1981; Tyler, 1?78; Teal, 1979; Sieber, 1974; Nash 
and Culbertson (eds.) 1977). 

Educational researchers, on the whole, possess doctoral degrees and have 
undergone formal training of considerable duration. This training was provided 
either by schools of education or discipline-based graduate departments such as 
psychology or sociology. In their professional settings, while some researchers work 
individually, others in small groups, all encounter *n environment reinforcing the 
research frame of reference. Researchers' work is formally communicated and 
frequently assessed by peers^through the usual mechanisms for professional, critique 
as in the assessments of proposals for funding, conference presentations, the reviews 
of books and articles. 

The reward structure of researchers includes, of course, the external rewards 

of employment and salary, but essentially the rewards of professional recognition 

\ 
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which are linked to increased access to research^ opportunities and ^ecasions for 
professional communication. Performance goals in research are necessarily very 
specifically defined and the assessment of goal realization is tied to scientific 
creativity and methodological skill. 

Again, a more precise picture will emerge if we briefly corjfeider specifically 
the components of frames of reference as we have done already in the case of 
elementary school teachers. 

4 (1.) Anchoring of the frame of reference in <*om rry tments and 
responsibilities: Researchers 1 reference frames are definitely anchored in a "code 
for inquiry" which they have been explicitly trained to employ through formal study 
and socialization. Their commitment is expected to be to problem-solving through 
means of replicable scientific methods. , 

(2.) Problem selection and formation: There are/ several ways in which 

/ 

research problems emerge or are presented to researchers. However, even in applied 
.settings the researcher technically exercises considerable latitude in the 
conceptualization of the problem. Questions to be answered or problems to be solved 
may arise from the evolution of the basic disciplines, such as research questions that 
are considered at the "frontier" in psychology; contemporary societal demand (for 
example new ways must be found to increase the scientific literacy of young 
children); or the individual researcher's curiosity or quest for invention (for example 
the desire to produce a "talking typewriter" to facilitate language learning). 
Whatever the source of the problem, researchers are responsible for formulating it in 
such a way that within some feasible scope of effort it can be presumed to be 
solvable. j 

(3.) Strategies of problem solution: We find enormous variety and great debate 
about appropriate strategies. However, while they may range from labortory 
experiments to field based observations, intervention studies, or qualitative 



Er|c * ' 395 



„ • 27 

■ ' 'f 

V * 

participant observations, aU strategies share a commitment to public replicabUity 
and therefore require frame of reference articulation. ~, • 
(4.): Rules of inquiry: Again we encounter very great variation of specific 
methodologies, and investigative styles. We have earlier in this paper referred to the 
frame of reference study (Holzner^Mitroff and Fisher, 1976) which is one illustration 
of the diversity of modes of inquiry used by social inquirers as well as educational < 
researchers. However, within the educational research community there is a 
normative emphasis on public argument in accord with empVical evidentiary rules 
and rational discourse. The ideal typical mode of inquiry of educational research is 
certainly the exact opposite of inquiry and problem solving that relies on purely 
experiential, personal, and private pragmatic validations. 

. (5.) Conception of the nature of facts: For researchers, the definition of 
"facticity" is relatively narrow. Informatiombecomes fact* if it is believed to have 
been generated through scientific, that is rcblicable, means. ^The contrast with 
elementary school teachers here is very strong: W example, scores of standardized 
tests are considered facts by .researchers but cjrtainly not always by teachers who- ' 
interpret all test scores through a filter of personal" observation and experience 
which gives the score its meaning to the teacher. 

(6.) Truth tests: Researchers' truth tests are 'similarly science based. In some 
cases, statistical validity can be established to substantia^ the venacity of a claim. 
Again, this is the very opposite of pragmatic modes of validation^ since the claim 
that "something works" is only the starting point rather than the end for research. 

(7.) Delimiting "boundaries: The boundary definitions of researchers tend to be 
of the disciplinary variety, as ii^ the fcase of the boundary of educational psychology, 
as well as of the problem focusStype. Thus, boundaries for frames of reference in 
research vary according to the problems to be solved, ranging from very precise and 
narrow boundaries for a program of laboratory experimentation, to more widely 
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drawn boundaries in the case of school-based program implementations. In any case, 
boundaries tend to be specifically drawn and are linked to judgments of knowledge 
relevance for research. A great deal of critical debate is focused on the boundary 
problem. 

(8. and 9.): • Articulation, codification, and reflectivity: As the work by 
Carolyn Hodges Persell which we cited a^bove illustrates there has been a conscious 
effort on the part or the leadership ', of the educational research community to 
increase degrees of articulation, codification, and reflectivity. As a consequence, 
the past two decades have been years of demonstrable change in the direction of the 
scientific professionalization of the educational research community. 

All of these aspects of the frames of reference in the educational research 
profession have direct implications for what knowledge gels used in the conduct of 
research work. The structuring of standards of judgment, truth tests, as well as the 
design of channels of communication, knowledge storage, and especially Critique, are 
powerful influences in determining what body of knowledge and pieces of information 
a researcher includes in his consideration of how to shape and solve the problem 
before him. * 
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Frames of Reference and the Prediction of Knowledge Use 

We believe that the preceding discussion? have shown good conceptual reason 
for the argument that knowledge of frames of reference can play a major role in an 
effort to understand and predict patterns of knowledge use. We have concentrated 
on the embeddedness of frames of reference in social organizations, that is in 
frameworks for knowledge. We have shown that the educational professions include 
widely divergent frames of reference and that, by implication, the linkage between 
research and the practice of teaching probably requires not only efforts at 
structuring linkage channels, organizations and roles, but also efforts at modifying 
the professional frames of reference to increase the knowledge reception capacity. 

It iS rather clear that individual frames of reference can be studied and 
described in detail, allowing the characterization of cognitive norms, preferences 
and directions that can be expected to be relatively stable for a given personality. 
Our focus was-at the level of social organization. We have pointed out that frames 
of reference are anchored in central role responsibilities and obligate a professional 
to maintain the "perspective" of his or her profession and to deploy the knowledge 
skills acquired through professional training, in turn, these cognitive activities of 
the professional rely on organized knowledge resources, be these of the informal kind 
as in networks of personal consultants, or of the formal variety as in the case of 
specialized libraries, knowledge synthesis services, information systems. Further, we 
have emphasized that the structure of the frame of reference is tied into a system of 
Critiquing and assessing knowledge claims, which in turn is usually articulated with 
the reward system and the channels of communication. 

At the same time our discussion has also shown that the educational professions 
are unequal in the degree to which they have developed adequate social frameworks 
for working, knowledge. Further, as an area of inquiry this domain has received 
insufficient empirical attention. Much of our evidence is of the observational and 
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anecdotal variety and many of the studies that do shed light on the questions we ask 
were actually designed for quite different purposes. Much work remains to be done. 

The area is important, because many of the strategies for knowledge use that 
have been employed in the past ignore^ the importance of frames of "reference. 
Certainly, the "two communities? 1 theory of knowledge use formulated by Nathan 
Caplan (1979) did improve the situation. However, our overview has shown that there 
are far more than two communities to be considered and that the consideration must 
be of a multdimensional variety. The frames of reference of "practitioners" 
certainly cannot be assumed to be terribly similar when we think of the enormous 
differentiation of the [Jfttctioner communities in the field of education alone— it 
should be noted that our illustration focused on elementary school teachers. We 
would have expected to find some significant differences if in addition we had 
focused on high school teachers, for example^ The systematic investigation of actual 
frames of reference in relation to knowledge should therefore be an important step 
in the planning of any major effort at improvement of professional practice and 
policy through the utilization of research based knowledge. 
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ABSTRACT 

This paper offers a, framework of methodological choices 
for studying knowledge use, choices which go much be- 
yond the selection of particular methods and techniques. 
As such, it attempts to^clarify the complexity in the 
field of knowledge use which has been conceptually soggy, 
theoretically fragmented and procedurally poor. Method- 
ological choices require the systematic consideration of 
alternative ways to conceptualize, define and explain 
knowledge use, as well as procedures that facilitate the 
acquisition of information required to address problems 
and hypotheses posed in accordance with prior conceptual 
and theoretical decisions. 
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METHODOLOGICAL CHOICES IN 
STUDYING KNOWLEDGE USE* 

William N., Dunn 
Program for the Study of Knowledge Use 
University of Pittsburgh 

This paper examines methodological choices in studying 
knowledge use. The term "methodological choice" is, intended, 
first of all, to call attention to opportunities for selecting 
new and more appropriate research procedures. There is now a 
large pool of potential- procedural innovations (see Dunn and 
Holzner, 1982) from which knowledge use researchers have scarcely 
begun to draw. Second, and relatedly, the term "methodological 
choice" is intended to convey a special concern with the reflec^ 
ive evaluation of research methods and techniques, rather than 
"technical" choices per se. Thus, methodology is about dome 
particular collection of methods and techniques; it is not it- 
self one of the collection (Watzlawick, Weakland, and Fisch, 



£aper presented at a Conference on Knowledge Utilization^ 
Theory and Methodology, Communication Institute, East-West Cen- 
ter, Honolulu, Hawaii, April 26-30, 1982. Portions of this 
paper were prepared under Grant No. NIE-G-81-0019 from the 
National Institute of Education, Research and Educational Prac- 
tices Program, U.S. Department of Education. 



1974) , 



r 



9 

ERJC 



4 06 



-2- 



The task of methodology is to raise to a level of self- 
conscious deliberation underlying assumptions that shape, often 
unknowingly, the selection of particular methods and techniques. 
Some of the most important questions confronting the study. of 
knowledge use are, in this sense, methodological. What do we 
mean by knowledge when we set about to measure its use (Larsen, 
1981)? Can we supply a persuasive definition of knowledge in 
general and, if so, does this general definition permit distinc- 
tions as to types of knowledge (Machlup, 1980)? What is the use 
of knowledge and f^w does it differ, if at all, from knowledge 
itself (Weiss, 1981)? Should the use of knowledge.be' defined 
in conceptual, instrumental, or symbolic terms (Caplan et al. , 
1975; Rich, 1977; Knorr, 1977)? On what basis can we < distinguish 
"ordinary" and "scientific" knowledge (Lindblom and Cohen, 1979)? 
Answers to these questions, since they depend upon information 
not supplied by methods and techniques themselves, are method- 
ological ones^. 

The main problem addressed in this paper may be stated as 
a question: How can we improve methodological choices in studying 
knowledge use? 4 In addressing this question the first section of 
the paper explores conceptual choices among alternative definitions 
of knowledge use as a phenomenon-to-be explained." This section, 
by specifying some eleven dimensions according to which knowledge * 
use may be defined, exposes the overly simple, narrow, and un- 
convincing character of many definitions now current in the field* 
The second section explores theoretic choices among alternative 
explanations of knowledge use. Here we consider two main types 
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of explanations that focus, respectively, on the effects of 
imposed versus generated phenomena. We a\Lso note some principal 



variations ^ithin types-^variations which j^ocus on charac- 
teristics of knowledge users of knowledge, ^nd^ bhe social sys- 
tems of which users are members. The third section examines 
procedural choices among alternative methods a\id techniques. 
This section attempts to evaluate the appropriateness of 
existing research procedures. The result i$r a m^tho<|JHU>gical 
conclusion; Some of the most widely used r^searc^ paocBdures 
fail to provid^ information required to assess the \empirical 
% import of various conceptual and theoretical choice^ outlined 
in preceding sections. The concluding section presents a 
brief for nrultidisciplinary innovation as a way to improve con- 
ceptual, theoretic, and procedural choices in the study of know- 
ledge use. 
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CONCEPTUAL CHOICES 



A basic question about any field of inquiry relates to 
definitions of its subject matter. At present the field of ' 
knowledge use is conceptually "soggy" (Weiss, 1977? 11), a - 
feature that makes it difficult to bound the field in convincing 
ways. As Ganz (1981: 186) observes, the fieTd is now hampered 
by a "continued adherence to an 'iron triangle' between creation, 
diffusion, and utilization without reference to the complex 
interactions among these activities, institutional arrangements, 
and social change..." Thus far, there is no plausible theoretical 
foundation for linking elements of this triangle, while the 
^tangled literature" (Nelson and Winter, 1977) of the field as 
a whole reflects the absence of a minimal set of criteria for 
defining knowledge use as a dependent variable (Zaltraan, 1980) 
or phenomenon-to^be-explained CDunn and Holzner, 1981K 

As the editor's introduction to the inaugural issue of Know- 
ledge; Creation, Diffusion, Utilization made* clear, the main 
reason for founding the journal was to provide a forum for linking 
individuals working independently on problems of knowledge creation, 
diffusion, arid utilization (Rich, 1979:,. 3) . ' * 

Under such conditions several alternatives are available to 
those who wish to organize., the field iri more coherent ways. One 



option is to elevate present difficulties to the status of an 
ontological principle, announcing that the "nature" of bureau- 



cratic organizations, the policy-making process, or of knowledge 
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itself imposes unalterable limitations on the creation of usable 
knowledge CLindblom and Cohen, 1979; Knott and Wildavsky, 1980) . 
The danger here lies in assuming that social systems are fixed 
entities whose establishment, maintenance, and change occur 
apart from the legitimating meanings and, purposes of their mem- 
bers. Alternatively, we may acknowledge the complexity of know- 
ledge use in contexts of practice, basing our views on ethno- 
methodological, phenomenological , or symbolic interactionist 
traditions (loosely, ^qualitative" methodology) according to 
which knowledge and itsr use are unique indexical expressions 
(Knorr, ,19821, Here we risk the abandonment ©^knowledge use 
as a v pseudo-problem" when, instead, it is an authentic problem 
whose generalizable dimensions we are unable to penetrate with 
tools now at hand. Finally, we may adopt on eclectic strategy, 
appropriating whatever concepts and models' that may be avail- 
able from existing social science disciplines and their sub- 
fields Csociology of knowledge., economics of information, cog- 
nitive psychology, cognitive ' anthropology! and hybrids (planned 
social change, social marketing, communication of innovations, 
information science, artificial intelligence*. Here the major 
risk is an unref lective form of eclecticism^ that may accelerate 
the conceptual entanglement of the field. 

Another alterative is to begin with the field as it is, 
taking conceptual diversity as a datum that may be organized 
in more manageable ways. This option not only permits a system- 
atic review arid assessment of competing definitions of knowledge 
use; it also precludes the premature adoption of favored definitions 
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that r on further reflection, may prove to be inadequate. Indeed, 
we will find that knowledge use is not a unitary concept and, 
for this reason, that it is wrong to assume that when we describe 
a particular type of knowledge use^-for example, conceptual or 
instrumental uses of scientific and ordinary knowledge — we are' 
in f^ct talking about different versions of the same thing tcp. 
Rescher, 19691. 

Dimensions erf Knowledge 

Taking its uses as giveV, knowledge r.ihayf be~conc£ptualized 
according to multiple dimensions that reduce to more orderly 
t£rmS the diverse meanings attached to the term: 

1. Subafcribership . Knowledge may be classified according 
to the persons or groups who subscribe to it. The dimension of 
subscribership includes such, categories as "personal knowledge," 

0 

"professional knowledge,* "practitioner knowledge," "public - 
knowledge," and so forth. In certain cases the use of knowledge 
may be virtually indistinguishable from subscribership for 
example, when practitioners Who subscribe to knowledge about 
managerial rules -of -thumb are also engaged, by definition, in 
acts of use which include the understanding and appreciation of 
that knowledge (Machlup, 1979:, 631,. Here knowledge and use are 
one and the same. Who subscribes to knowledge? XJnder what con- 
dition is the act of subscribing to knowledge identical to using 
it? * ^ 
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2. Source . Distinctions may also be basal? on the source 
of knowledge. Distinctions by source include "scientific 
knowledge," "professional knowledge ," "craft knowledge," 
"practice knowledge," "experiential knowledge," "ordinary know- 
ledge," Social scientists may be the 'source of knowledge while 
others subscribe to it, long after the same social scientists 
have disavowed their original claims. Conversely, social 
scientists may subscribe to knowledge whose source is practical, 
experiential, or ordinary. Source and subscribership should not 
be confused, since much "scientific" knowledge (subscribership) 
is in part "ordinary" knowledge (source J (see, e.g., Lindblom and 
Cohen, 1979L. From what source did knowledge arise? Is the 
source different from the subscriber? 

3. Object . Knowledge may also be classified according-to 
its object. Distinctions by object include "economic knowledge," 
"political knowledge,", •environmental knowledge," as well as 
knowledge whose objects are particular policy issue areas or 
sectors— for example, criminal justice, housing, welfare, edu- 
cation, and so forth, The use of knowledge tends to vary with 
the complexity of objects, such that some types of knowledge — 
for example, knowledge about experimental programs undertaken 

as part of the Great Society (see Aaron, 1978} seem to affect 
policymaking more than others^"- for example, knowledge about the 
criminal justice system (see Wilson, 1978). The object of 

o 

V 

knowledge is independent of its subscribership and source. What 
is the knowledge about? Are different types of knowledge, clas- 
sified by object, associated with use? 
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4. Benefit , Knowledge may also be classified in teOT^s 

of the types of benefits expected to occur upon its use. Machlup 
(1980) , for example, distinguishes knowledge according to 
expected benefits which^are "practical , " "intellectual," "pas- 
time, " "Spiritual f M and "unwanted." The expected benefit, of 
^knowledge is not the same as its object, since knowledge about 
a particular ohject-r-e, g t , the politics of incremental policy- 
making^inay confer benefits that are primarily intellectual Cor 
spiritual! and only secondarily, if at all, practical. More- 
over, the dimension of benefit is independent of subscribership 
and source. Professional knowledge Csrubscribership) based on 
experience Csourcel may be primarily intellectual or pastime 
knowledge (benefit) insofar as there is no expectation that its 
use will improve practice^Erureaucratic routines, for example, 
are often Valued intrinsically rather than for their conse- 
quences. In either case, ^however, it makes sense to talk about 
the "use" of knowleSge, What difference will the use of know- 
ledge make? 

5, Purpose . Knowledge may be classified according to the 
type of purpose served by the realization of a benefit. Habermas 

0 

CL9711, for example, distinguishes purposes of "control," "under- 
standing,* and "emancipation,* "Practical" knowledge Cbenefit) 
does not necessarily expand capacities for controlling the human 
and material environment (purpose}, while "intellectual" and 
"spiritual" knoweldge Cbenefit \ does not necessarily enlarge 
capacities for understanding and emancipation (purpose 1 . In 
many cases the realization of practical benefits by using knowledge 
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about social problems (race relations, sxibstance abuse, rape, • 
delinquency, suicide, alienation), seems to .have distorted our 
understanding of problems, as experienced hy victims, and 

> 

depressed capacities for emancipating social scientists and 
practitioners from tacit assumptions and theories that impair 
the realization of human potential (sejs Gregg vet al., 1979). 
Moreover, the purpose of -knowledge is independent of its sub- 
scribership, source, and object. The professional knowledge 

of successful managers Csubscribershipl , based largely on prac- 

x • . 

tical experience CsourceL with, complex organizations (object) , 
m^ be oriented as ^Ruch towards Tinde£standing>the perspectives 
of organizational members and challenging their tacit assumptions 
as it is towards technical control; Apart from expected or 
Actual benefits of using knowledge, does the realization of such 
benefits serve purposes of technica^ control? Of human, under- 
standing? Of emancipating individuals and collectivities from 
hidden assumptions that impair reflective choices? 

6. Warrant , Knowledge may be classified according to the 
type s~ of assumptions that wartant its certification as know- 
ledge (Toulmxn, Rieke, and Janik, 19791. Knowledge-warranting 
assumptions may be empirical, analytic, teleological , pragmatic, 
authoritative, ethical, pragmatic, and so forth. Warrants for 
knowledge ("reality tests*! are organized in frames of reference 
or cognitive maps^that affect and are affected by social frame- 
works (Holzner and Marx, 1979; Weiss and Bucuvalas, 1980). 
These warrants represent standards, criteria, or rules for 
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assessing the adequacy, relevance, and cogency of knowledge 
'claims (Dunn, 1982}. The type- of warrant used to certify 
knowledge is potentially independent of its subscribership, 
source, object, benefit, and purpose, "Practitioner knowledge" 
tsubscribership) may be based on the/^same kinds of warrants 
employed by social ^scientists; and tLere is no necessary relation- 
ship between the source of knowledge Ce.g., every day exper- 
ience} and its theoretical adequacy or practical relevance 
(Zaltman ert al . , 19821, The expected benefits and purposes of 
knowledge would appear to be a function of the knowledge-war- 
ranting assumptions of different subscribers. What makes some- 
thing count as "knowledge" in the first place? 

Dimensions Q^lJse 

Once we have chosen ^n appropriate definition of knowledge, 
an effort that surely will involve several dimensions outlined 
above, we can proceed to specify what it means to say that such 
knowledge .is "used," This task may seem relatively straight- 
forward , since we can simply look at the decisions ma'de by users — 
for example, ^olitymakers, managers,, clients, social scientists, 
members of society-at-large-— and determine whether such decisions 
are based on a particular type of knowledge. Yet the concept 
of use, like that of knowledge, is by no means a unitary one; 
users, to paraphrase Carol Weiss, do not utilize knowledge the 
way they utilize a hammer (Weiss, . 1981: 18). Hence, use may 
also be conceptualized according to multiple dimensions: 
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( 1. Usership , f Taking the content of knowledge as given , 

use may be classified according to the persons or groups who 
constitute users of knowledge. Users may be national or local 
policymakers,., members of professional associations, media repre- 
sentatives, client groups, social scientists, and so forth. 
While it is difficult to imagine users who are not also sub- 
scribers, it often happens that some users subscribe to knowledge 
for reasons that differ from those^of subscribers— for example, 
policymakers may use results of evaluation research to support 
their own personal or political ambitions, thus subscribing to 
knowledge for reasons that are quite different from those of 
program evaluators. Under these -conditions we might speak 
about the "misutilization" of evaluation research (Cook, ' Levinson- 
Rose, and Pollard, 19801. Apparently simple issues such as 
this one are difficult or impossible to resolve, it seems, 
without also addressing the expected benefits, purposes, and 
, warrants of knowledge, Who are the users of knowledge? 

2. Object . Use may also be classified according to its 
objects, which may include recommendations, empirical general- 
izations, hypotheses, theories), models, concepts, assumptions, 
principles, ideas, and so fiorth^_ The object of knowledge use 
and the object of knowledge dif f%r^since^ knowledge classified 
by object Ce.g., '"political" knowledge) implies nothing . about 
the object of use (e.g., policy recommendations versus political ' 
ideas}. Moreover , 1 choices involving different objects of use 

would appear to depend on -variations in expected benefits and 

v . 
purposes of knowledge, as well as. knowledge-warranting assump- 
tions. What is being used? Why? 
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3. Directness . Use may be classified according to its 
directness to an original knowledge source. Use may be relatively 
direct, in cases where some user reads an original report or 
study produced by a social scientist, or indirect. Indirect 
use occurs when research findings are filtered through executive 
summaries and secondary written descriptions or reviews, or 
passed on through conversations that involve no face-to-face 
Interaction ("invisible colleges*! . How close is the use of 
knowledge to its original source? 

4 « Temporal Proximity . Use may be classified according 
to its proximity in time to an original knowledge source. 
Apart from its directness, use may be immediate, occurring 
simultaneously with the creation of knowledge— for example, 
when subscribers are also the source of knowledge they would 
appear to engage in immediate use. Use may also be delayed, ' 
as when knowledge is transmitted across year's, periods, gener- 
ations, or historical epochs. The concept of "diffusion time" 
is a specif ic variant of temporal proximity, as is Keynes 1 
statement that policymakers are the slaves of some defunct ' 
philosopher. How immediate is the use of knowledge? 

5 « Magnitude of Expected Effects . Use may also be clas- 
sified according ^to the magnitude ot its expected effects. Use 
may refer to "conceptual" and/or "instrumental" effects, cate- 
gories which appear to be points along a. single continuum or 
dimension, rather than discrete and mutually exclusive classes. 
In this context, Machlup U98QI describes thirteen elements of 
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the state or act of knowing whicfi specify, in part, what is 
meant by the magnitude of expected effects of use. These 
elements include acts of knowing that range from being 
acquainted with or aware of something to being able to explain, 
demonstrate, talk about, or perform some action. One of these 
effects (i.e., the acquired capacity to perform some action)* 
is clearly n instrumental , * since it implies action, while 
others seem to be "cgnceptual. " None, strictly speaking, are 
behavioral. Yet being able to explain or demonstrate some- 
thing — for example, explaining or demonstrating that sodfcal 
experiments have certain effects«~is instrumental to being 
able to perform and actually taking actions that will alleviate 
some social problem. For this reason, the expected effects of 
using knowledge are best CQncep totalized as a hierarchy of 
requisites similar in form to a unidimensio/ial (Guttman) scale 
whose poles are being acquainted with something and actually 
performing some action. Finally, the concept of "symbolic? 1 

use (JCnorr,. 1977) appears to refer to the expected benefits 

> 

and/or purposes of knowledge,, rather than to the magnitude of 
expected effects per se. How much effect is required to count 
as "use?" 
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THEORETIC CHOICES 

The striking feature of these eleven dimensions, when con- 
sidered as a choice set from which one may select definitions, is 
that few questions about knowledge use may be answered without 
making reference to the internal cognitive states of users. 
Simple questions about subscribership, a usership, and source > as 
well as more complex issues surrounding benefits, purposes, 
warrants, and expected effects, are difficult or plainly impossible 
to answer by reference to overt behavior alone. For this reason 
it makes much sense to conceptualize knowledge use in terms of 
"historical semantics, trying to find out what people mean (or 
meant) when they say (or said) something, not what they ought to 
mean (or ought to have meant)" (Machlup, 1980: 38). At the same 
time the overt behavior antecedents and consequences of knowledge 
use — antecedents and consequences which involve individual as 
well as collective behavior — miist also be addressed. 

Alternative Classes of Phenomena-to-Be-Explained 

In order to outline theoretic choices we must first attempt 
to provide answers to two questions (cp. Wallace, 1969) : (1) How 
is knowledge use to be defined ? (2) How i^ knowledge use to be 
explained ? The first questiorj pleads towards alternative defini- 
tions of knowledge use as a phenomenon-to-be-explained (explanan- 
dum) , while the second yields alternative classes of phenomena 
available to 'explain knowledge u?e (explanantes) • By pursuing 
both questions at once we may avoid "difficulties that arise vfhen 
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two or more theoretical \erspectives based on a cqm^fn definition 
of knowledge use — for example, a definition in terms of expected 
instrumental effects — also offer fundamentally different expla- 
nations. The -same situation may arise in reverse when two or 
more theoretical perspectives are based on a similar explanation , 
of^entirely different phenomena. Accordingly, conflicting research 
findings and conclusions are not due solely to primarily to an 
absence of definitions ^f use, or even to a failure to employ « 
multiple measures of the same phenomenon (Larsen, 1981: 151), but 
'to the presence 'of^ixed and/or conflicting definitions and expla- 
nations of knowledge use. 

Multiple dimensions of ^knowledge use reviewed in the last 
section seem" to share f one property in common : Knowledge use is 
a cognitive relation.^ By specifying that knowledge use is a 
"cognitive" relation we mean that the accompaniment of one actor's 
behavior by that of another is somehow based on the act or 
faculty of knowing. Indeed, it is this cognitive stipulation, a^d 
perhaps this alone, that differentiates knowledge use frdta disci- 
plines or fields that investigate other types of relations based, 
for example, on power, status, or exchange. Accordingly, any 
satisfactory definition o^ knowledge use would appear to hinge 
on the cognitive properties of "knowers" or "users" (see Churchman, 
1971) . 

As a cognitive relation knpwledge use may be defined in two 
fundamentally different ways: one type of definition emphasizes 
overt and directly observable cognitive relations, while the 
other stresses cognitive relations which are covert and indirectly 
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observable. The difference between these definitions, as Wallace 
(1969: 6-7) observes of social relations in general, 

corresponds to the difference between the Newtonian 
and Weberian definitions -of "action. " Newton «s defi- h 
nition of mechanical action was entirely in terms of 
external observables. . . Weber took exactly the oppo- 
site view by asserting that bodies' social action is 
so far distinguishable from mecnanical action as to 
be definable only in terms of their internal states. - 

The distinction between covert. and covert relations is, of course, 
somewhat ambiguous. One source of ambiguity is the technical 
difficulty of directly observing subjective states (e.g\ , know- 
ledge, attitudes, values). - Another technical difficulty, how-r 
ever, is determining whether overt behavior does or does not . - 
reflect particular cognitive properties *~ for example, kndwledge 
of policy recommendations or the findings of program evaluations . 
For this reason Mert'on (194$) , Suchman >1972) , Weiss (1977), and 
Rein and White (1977) have called attention the the ."latent" 
goals of policy research^ Often it is even more difficult to 
make inferences about "knowledge use" from overt behavior than 
it is from subjective states. , • 

^ The definition of knowledge use as a covert cognitive rela- 
tion is evident in the literature of the field. House (1981: 12-13) 
for example, recommends ^that investigations of knowledge use in 
education be "directed at the different 'meanings' produced by ' 
the change efforts rather than at the change itself," citing an 
example where an unsuccessful attempt to introduce PPBS was 
accompanied by extensive culture conflict (see Wolcott, 1979). 
Similarly, Aaron (1978: 159) stresses, that the use of research 
and experimentation on poverty, while it is seemingly unconnected 
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to overt changes in. the behavior of policymakers, nevertheless 
"corrodes" the king of simply faiths on which" political movements 
are built." In the same^vein, Patton's work on utilisation- 
focused evaluation defines knowledge use in terms of "more pene- 
trating perspectives, increased .capabilities , and greater coramit- 
ments to action" (Patton, 1978:|290). Caplan, while acknowledging 
the importance o£ "instrumental K use , stresses the importance of 
covert cognitive relations found tc/ characterize "conceptual" m 

■ — 

use among high-level policymakers confronted by complex macro- 
level problems (Caplan, 1979: 465,) . ^^^^ 

In contrast to an emphasis on covert and indirectly obser- 
vable cognitive relations we find those who stress relations that 
are overt aifd directly observable. Here there is no direct 
reference to underlying purposes, benefits, or knowledge-warranting 
assumptions. Thus, for example, knowledge use has been viewed 
as an overt process of adopting and implementing research-based 
innovations (National Institute of Education, 1978), or as a 
change in the structure and functioning of a social system 
(Rogers, 1973). These definitions of knc^/ledge use, which empha- 
size directly observable cognitive relations, assume that use 
occurs when an entire set of recommendations is implemented in 
a form suggested by researchers (Larsen/ 1981 : 150). 

Another definition of knowledge use in termj> of overt cogni- 
tive relations is provided in a study of the use of evaluation 
research in criminal justice* Noting that "evaluation is a pro- 
cess that produces information to assist in the allocation of 
resources," the authors go on to define use in terms of the 
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"decision-consequential impact" of research (Larsen and' Berliner, 
197^9: 2). Quoting Howard (1966), the authors contend. that "a 
decision is net a 'decision to make a decision, ' but rather the 
concrete action implied ly the decision" (Larsen and Berliner^, 
19,79 : 23n) . This definition, and others like it, provider 
evidence that the term "overt cognitive relation" is not a self- 
evident contradiction. While we may disragreq . with such definitions 
it is important to recognize that many Observers assume (often 

implicitly) that overt and directly * observable behavior indicates, 

r 

^suggests, or implies acts of knowing. j 

The distinction between overt and covert cognitive relations 
provides a simple bases for comparing and contrasting \\ in gross 
terms, alternative definitions of phenomena-to-be-explaiit^d . 
Clearly, however, dimensions of knowledge use reviewed in ttfc 
last section pe^ait many additional distinctions, many of which, 
have been made in available literature (e.g., ^arsen/et al . , 1976> 
Wefss, 1 1976 ; Hall ani Loucks , 1977; Caplan 1979 ; Dunn, 1980; 
Knott s^fWildWsky, 1980; Zaltman, 1980; Rich and Goldsmith, 
1981) . Perhaps most appropriate position towards these opportuni- 
ties for choice is simply to argue that the overt-covert distinc- 
tion captures many of the most important definitions current in 
the field. The overt-covert distinction may help clarify essen- 
tial properties of more complex classifications. For example, 

f ■> 

Knott and Wildavsky (1980: 546) propose a seven-fold typology of 

"standards of knowledge utilization," some of which (e.g., cogni- ^ ■} 

J ' 

tion and reference) conform to what we Kave defined as covert and 
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indirectly observable cognitive relations, while others (e.g., 
implementation and impact) are instances of relations that are 
overt and directly observable. „♦ • 

Alternative Classes of Phenomena that Explain 

Apart from their different emphases on overt and overt cogni- 
tive relations, theoretic choices may involve different classes 
of explanatory phenomena. Theoretic choices made on the basis 
of some common definition of knowledge use as a covert cognitive 
relation (e .g. , "conceptual" use) frequently involve different 
explanations of this same phenomenon — for example, explanations 
derived from kupwledge-specif ic, policymaker constraint, or- 
two-communities theories of knowledge use (see Caplan, Morrison, 
and Starabaugh, 1975). A similar situation may exist in reverse, 
since' authors who share ^e same explanatory schema — for example, 
an interactive, pluralistic, and incremental theory of policy- 
making — may hold fundamentally different defintions of knowledge 
use; Thus, while Lindblom and Cohen (1979: 12) affirm that 
"knowledge is knowledge to anyone who takes it as a basis for 
action," Wildavsky (1979: 27-28) and Knott and Wildavsky (198P) 
reserve the term "knowledge" for ' 

...theories relating policy variables to effects where -~" 
the principles are confirmed by the empirical test of 
repeated decisions... When policy makers are certain 

that manipulating these variables will produ<fie - EEi^ 

expected effects — that is, if "x" is done, !"y» will 
follow with a known probability — then they have know- 
ledge (547, 546. Emphasis supplied.). 
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Whatever el.se one may say about these two ifftteractionist explana- 
tions of knowledge use, their authors hardly share a common 
theory -- they attempt to explain entirely different phenomena. 

The principal phenomena off fered to explain knowledge use may 
be grouped into two main classes: conditions that are imposed on 
users by the given nature of knowledge , of users themselves, or 
of the social system of which they are members; and conditions 
that are generated b£ users who create knowledge, develop their 
own standards for assessing knowledge, or who participate in 
social systems where knowledge is produced. The first type of 
explanation implies that knowledge use is "natural," "determined," 
or "imperative," while the second suggests that knowledge use is 
"artificial," "sfelf-determining, " or "constructed." 

Theoretic choices stressing imposed phenomena are sometimes 
made- by those concerned with the nature of social science know- 
ledge and its effects on research utilization. For example, 
Bernstein and Freeman (1975), observing that "there have been few 
cases of actual effective utilization of evaluation research for 
expected purposes" (5), offer an imposed explanation based on 
assumed characteristics of "quality" research and the attendant 
failure of federal agencies to ensure its production. "Quality" 
research is defined as research that conforms to standards of 
quasi-experimental design (10Q-101) , while the failure to enforce 
such standards is described in the following terms: 

...there is neither a federal evaluation policy nor a 
set of requirements ;and guidelines regarding what con- 
stitutes aji appropriate evaluation. This lack has 
severe consequences. It results in a failure to enforce 
any standardized set of evaluation requirements, even 
when present in legislation (Bernstein and Freeman , 
1975:. 6) 
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Whereas Bernstein and Freeman explain knowledge use in terms 
of Imposed properties of knowledge, others stress explanations 
based on properties that have been imposed on users or their 
environing social systems. For example, Mitroff and Mitroff (1979) 
employ Jungian personality theory, which assumes given underlying 
personality structures or traits, to explain patterns of inter- 
personal communication among knowledge users and producers. ' 
.Whereas this explanation focuses on imposed user-specific charac- 
teristics, Others stress phenomena that operate through social 
systems. For example, the weight of arguments offered in support 
of theories of a "post-industrial" society (Bell, 1976) is borne 
by explanations that stress imposed system-specific characteris- 
tics — particularly, the claim that the growing interdepehdencies , 
complexity, and pace of change of contemporary society make 
existing knowledge obsolete, thus increasing the demand for new 
forms of policy-relevant knowledge (see Straussman, 1976). 

Theoretic choices are also based on explanations that empha- 
size phenomena which are generated by users as they create know- 
ledge, develop their own standards for assessing knowledge, 
participate in social systems which produce knowledge. For example, 
Carol Weiss outlines three classes of generated phenomena which 
parallel distinctions among knowledge-specific, user-specific, 
and so«i»a^system-specific characteristics: knowledge -driven, 
decision-driven, and interactive models of knowledge use (C. Weiss, 
1977: 11-14) . In her own work Weiss has offered explanations 
based primarily on phenomena generated by users — specifically, 
the truth tests and utility tests that constitute the "frames of 
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reference of decision makers (Weiss and Bucuvales, 1980). Where- 
as Lindblom and Cohen (1979) are concerned with knowledge-specific 
characteristics in general — that is , with relations between 
"ordinary" and "scientific" knowledge — Weiss and Bucuvales 
specifically address knowledge as a product of user-specific 
frames of reference. That they are not principally interested 
in social system-specific characteristics is evident^in their 
description of their enterprise as a classical sociology of know- 
ledge "turned upside down" (see Holzner, 1978: 8): 

Whereas the classical sociology of knowledge was 
concerned with tl\e social bafcfes of intellectual ^ 
productions, a sociology of knowledge application 
would be concerned with the social consequences of 
knowledge... /there is a/ need for understanding 
the multiple Frames of reference with which actors 
perceive knowledge and the discrepancies between the 
frames of reference of knowledge producers and 
knowledge users (Weiss and Bucuvalas, 1980: 302). 

Finally, other explanations place pr^^mary emphasis on pheno- 
mena generated by social systems. Janet Weiss argues that atten- 
tion to "relationships between the structures of policymaking 
and the avenue^ by which social science may participate ih .policy 
debates permits analysis of both the overall level of social 
science use that sectoral arrangements -sg&n to support and the 
particular Jc^ds of use that sectoral arrangements encourage" 
(J. Weiss, 1979: 439). Caplan provides a similar explanation by 
stressing generated phenomena that also operate through social 
systems: "social scientists would be well advised to pay particu- 
larly close attention to the utilization theories that stress 
the lack of interaction between social scientists and policymakers 
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as a major reason for nonuse" (Caplan, 1979: 561; see also Caplan, 
Morrison, and Stambaugh, 1975; and Poppen, 1978). 

A Basic Typology 

The intersection of definitional and explanatory dimensions 
outlined above yields a basic typology of theoretic choices 
(Figure 1). The product of the typology is four ideal types 
labelled with terms that describe the essential features of com- 
peting definitions (explananda) and explanations (explanantes) , 
as these have been discussed so far, rather than disciplinary 
orientations (e.g., '-'philosophy of science" or "sociology of know- 
ledge") or preferred metaphors (e.g., "two-communities theory"). 

One major set of theoretic choices may be designated "beha- 
vioral imperativism, " which defines knowledge use in primarily 
overt' (behavioral) terms and offers explanations that stress 
phenomena that are imposed on users (imperative) . The work of 
Knott and Wildavsky (1980) closely approximates "behavioral^pera- 
tivism," as we have characterized this perspective, since their 
theory of dissemination and knowledge use holds that the overt 
and observable utilization behavior of policymakers is a function 
of the availability of "authentic" knowledge developed through 
"natural" processes. Specifically, \ 

a 

— premature dissemination in the absence of knowledge 
contributes to information overload, thus making dis- 
semination of cause of underutilization rather than a 
cure. Dissemination should not substitute for supply 
when natural processes are cheaper and more effective 

Three difficulties stand out : pretensions to 
knowledge, which make many dissemination ef forts pre- 
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Figure 1 

A BASIC TYPOLOGY OF THEORETIC CHOICES 



The principal Cognitive Relations 
that Define Knowledge Use Are: 



\ 



The Principal 
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mature; artificiality (or the Newcastle syndrome) sub- 
stitutes unnecessarily for the natural spread of know- 
ledge; and the Mathew effect in which those policy 
) makers who are the most in need of dissemination are 
also the ones who are the least able to process and 
apply what is sent (Knott and Wi-ldavsky, 1980: 573, 
574). • % # , 

A second set of choices, "conceptual imperativism, H defines 
knowledge use in essentially covert (cdnceptual) terras and pro-* 
vides explanations that stress phenomena that are imposed on 
users (imperative). Machlup (1980) approximates conceptual im- 
perativism, as we have defined it, since the emphasis is on a 
"knowledge industry" constituted by given structures of produc- 
tion and occupational specialization whidfh create new and divers^ 
forms of knowledge. While "for most parts of the production of 
knowledge no possible measure of output can be conceive^that 
would be logically separate from a measure of input" (MacKl\Ip\ 
1980: 225), the class of imposed phenomena used to explain know- 
ledge and its use are based on 

an industry approach to the total value of the output 
of all firms, agencies, departments, etc., that gen- 
erate or disseminate knowledge, and an occupation 
approach to the amount or value of the input of know- 
ledge producing labor (Machlup, 1980: 227). 

A third choice set, "behavioral constructivism," defines 
knowledge use in primarily overt (behavioral) terms and supplies 
explanations that stress phenomena that are generated by users 
(constructed) . Behavioral constructivism is evident in explana- 
tions that stress "the symbiotic relationships among policy actors, 
knowledge, and the political and institutional context of policy 
making" (J. Weiss, 1979: 456-57). The use of knowledge by 
policymakers 



431 



cannot be understood independently of the processes 
for making public policy. These processes provide 
the critical organizational and political contexts 
that shape policymakers • needs for outside exper- 
tise; create organizational channels of information 
flow that routinely bring (or fail to bring) ideas 
and evidence to policy makers' attention and define, 
problems, decision situations, policy alternatives, 
and solutions in ways that determine which evidence 
is considered to be relevant (J. Weiss, 1979: 439). 

The fourth set of theoretic choices, "conceptual construc- 
tivism, - is best represented by the work of Knorr (1981). Concep- 
tual constructivism defines knowledge use essentially covert 
(conceptual) terms and offers explanations that stress phenomena 
that are generated by users (constructed). For Knorr, contexts 
of practical action are radically under -determined by generalizable 
standards or rules. As such, practical action is indexical: 

...rules and decision criteria, and more generally 
definitions of the situation, are interpreted in 
con text... it is the concrete, local translation- 
of rules or decision criteria which determine 
the selections that are made, and which subse- 
quently shape the outcomes of these selections 

(Knorr, 1981) . 

Clearly, these four sets of theoretic choices are highly 
general ones; they do not capture the range of particular assump- 
tions, propositions, or hypotheses put forth in alternative 
theories of knowledge use. For this reason it is desirable to 
differentiate the basic typology into additional subclasses of 
explanations. This yields a twelve-fold typology based on addi- 
tional distinctions among knowledge-specific, user-specific, and 
social system-specific phenomena (Dunn, 1981). 
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PROCEDURAL CHOICES 

«* 

Procedural choices involve the selection, adaptation, or ) 
development of research metfiods and techniques. Procedural 
choices, however, are dependent on prior conceptual and theoretic 
decisions. The choice of a particular method (e.g. , ethno- 
graphic interviews) or technique (e.g., multidimensional 
scalingl represents a means for op^rationalizing prior conceptual 
and theoretic choices. A key methodological issue is whether 
available research procedures provide information required to 
explore and/or test hypotheses that may be derived from the 
basic typology of theoretic choices presented in Figure 1. 
Another important question is whether available procedures 
yield information about the eleven dimensions of Jcnoweldge use 
discussed in the first section of this paper. Are given pro- 
cedural choices appropriate for the kinds of conceptual and 
theoretic decisions that m^y Be made in conducting research on 
knowledge use?. 

Av^ilab^e research methods and techniques for studying 
knowledge use are largely inadequate. This is one of the main 
conclusions of a methodological research project on knowledge use 
and school improvement that is Being carried out with the sup- 
port of the National institute of Education. This project, 
among its other objectives, has involved an extensive search 
for research procedures that may facilitate improvements in 
knowledge use research carried out in areas that range from 
education and mental health to criminal justice, local govern- 
ment, and the social and economic development of Third World 
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countries* The main product of this research effort is a volume 
(tentatively! titled Handbook of Knowledge Utilization: Assessing 
the Impact of Research and Development in Education and the Social 
and Behavioral Sciences . Major portions of the volume are avail- 
ab^Le in draft from and its table of contents is appended to 
.this paper. 

The approach to developing the handbook has involved several 

kinds of activities. Since one of our aims is conceptual and 

theoretical, we have invested a good deal of effort in reviewing, 

assessing, and attempting to develop alternative conceptual 

frameworks and models, yet we are also concerned with methodology 

and, for this reason, we have also reviewed, assessed, and 

attempted to develop alternative research methods jand techniques. 

In reviewing available research procedures we began With 

traditional literature reviews. Concurrently, we consulted 

^with members of several overlapping research networks (NIMH, 

NSF, NIE) , asking them to provide copies of questionnaires, 

interview schedules, coding schema, analytic routines, and so 

* 

forth. In addition, we reviewed available compendia of research 
procedures in the social and behavioral sciences and searcHted a 
major archive of instruments to measure creativity and innova- 
tion. Altogether, we attempted to contact more than 100 researchers 
in the field and acquired, directly and indirectly, more than 
200 "candidate procedures. * We eventually narrowed this larger 
set to some 84 procedures that include content analysis, question- 
naires and interview schedules, observational schema, tests, case 

analysis routines, and scales and indices (see Appendix, Part III) . 
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Thus far, we have used two criteria for Including procedures 
in the handbook: reproducability and face relevance to knowledge 
use. The criterion of reproducability required that a given 
procedure be sufficiently specific, regular, and orderly ,tha# s 
its steps could be repeated by some other researcher. * By using 
this criterion of inclusion we lost a large number of "candidate 
procedures;" much published literature on knowledge use does not 
meet this standard. In some instances— especially , those involving 
"qualitative" methods— researchers explicitly challenge- or dis- 
avow the appropriateness of reproducability as ^criterion. 

The second criterion— face relevance to the study of know- 
ledge use — required judgments about the conceptualization of 
the field. Here we were guided by the general consideration x 
that .procedures should permit the acquisition of information 
about multiple dimensions of knowledge use; that procedures 
should be appropriate for investigating alternative theories; 
and. that procedures should somehow tap cognitive properties, 
since it is this feature that provides ^the field with a unique 
mission and identity. The criterion of face relevance,, when 
applied to research on knowledge use proper, resulted in the 
inclusion of some 46 procedures; an additional 38 procedures 
were included as^relevant, even though they had not been developed 
or used by researchers who consider themselves part of the field. 

Although we are still in the process of selecting procedures 
for inclusion in the handbook, it is possible to offer some pre- 
liminary methodological findings: 
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!• Repr o due abi 1 i t y ; One of the striking features of 
-pjiblished and unpublished research is the relative 



paufeirty: of reproducible procedures. For example, 
many case studies and interviews are not based. on 
steps that are sufficiently definite, regular, or ' 
orderly to be^repi^ca^ed by others in the same of 
different contexts. 

2. Replicability ; Even where procedures are reproducible, 
there are few instances where the same procedure has 

* been used by two or more investigators. Ironically, | 
knowledge use researchers* seem to use knowledge gen- 
erated in the field no more than colleagues in other 
disciplines and fields. Individual investigators 
develop and use their own procedures; consequently, N 
there is little cumulative growth of research findings. 

3. Conceptualization ; More thah onp^half of studies based 
on reproducible procedures define knowledge use con- 
ceptually, or both conceptually aindv instrumentally i 

All retraining studies define use instrumentally and, 
aimong these studies, none focus on the expected bene- 
fits, purposes f or fenowledge^warranting assumptions of 
users themselves, .Among studies based on a conceptual 
definition of use, few address expected benefits-r- 
purposes, or warrants. Only 8 of 42 studies focus on 
underlying assumptions or meanings of users. For this 
reason, few studies Cat least those based on repro- 
ducible methodsl *re "qualitative" in the specific 
sense that they address questions surrounding the sub- 
jective interpretations of psers* 

4 < Units of Arialysis , Individual users are the predominant 
unit of analysis; collective properties are inferred 
by aggregating individual data. Few procedures are 
available for mapping collective cognitive properties. 
While the term "network" is part of the basic vocabulary 
of the field, there appear ,to be few applications of 
network analysis as a set of reproducible procedures 
for mapping cognitive relations among u^ers/ ("epistiemic 
networks"!-. 



5t Reliability and Validity ; More than one-half of the 

are appropriate re- 
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ne-Liaoixixy ana vaxxaity ; nore tru 
studies for which reliability data 

port such d&ta. Nearly all studies thab^employ testing 
and content^analytic procedures report reliability data; 
less than half of studies using questionnaires and inter- 
view schedules report reliability data/ Overall, there 
have been few attempts to establish tMe validity of ' 
constructs, even though such prdceduijps — since they are 
specifically designed to examine the status of mental + 
constructs of all kinds, including cognitive ones — are 
clearly central to the study of £jiowledge use. c 
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At present the field of knowledge use is conceptually soggy/ 
leoretically fragmented, and procedurally poor. This paper has 
attempted to supply a framework of methodological choices for 
studying knowledge use, choices whjfccfl *go much beyond the 
selection of particular methods and techniques. Methodological \ 
cl^oices require the systematic consideration' of alternative* 
w^ys tt> conceptualize, define, and explain knowledge use, as 
w^ll as procedures that facilitate the acquisition of inform- 
ation, required to address problems and hypotheses posed in accori- * 

ar^ce with prior conceptual and theoretic decisions. 9 * 
i . * 

I By attempting to enlarge the doroiin o£ methodological choice 
tljiis paper may have produced a greater sense of the complexity 
o^ studying knowledge use, hopefully, by clarifying rather than 
cdmplicating^this complexity. A paper of this kind does not 
justify conclusions not already made in the text. Accordingly, 
it may he appropriate to end the pap6r with a plea for methodological 
innovation. We need to adopt, adapt, reinvent, and institutionalize 
Concepts, theories, and procedures developed ^elsewhere and among 
Ourselves, A familiar theme. 
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